2024-04-26T19:44:11.819 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T19:44:11.819 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138 --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/orig.config.yaml 2024-04-26T19:44:11.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T19:44:11.954 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138 branch: wip-rishabh-testing-20240426.111959 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: null first_in_suite: false job_id: '7675138' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGwX4eQ/wpTKEGvn18+qTf2oYqdCO1H0iRhw2c0Gwmi3Hw2hrx6eYf4A0MKCXS3N6up0unUau9jfXFdbuE8VA0= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzPW8O5pJzgxy81++2UbljKm0/l8yFx5Z+2tD0wPo7w02Ek+rPl0d2yprJHBgcPk27ea05/YSa6Dt/2oVn9H4E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes - tasks.cephfs.test_subvolume.TestSubvolume teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T19:44:11.954 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T19:44:11.955 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T19:44:11.955 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T19:44:11.956 INFO:teuthology.task.internal:Checking packages... 2024-04-26T19:44:11.983 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T19:44:11.983 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T19:44:11.983 INFO:teuthology.packaging:ref: None 2024-04-26T19:44:11.983 INFO:teuthology.packaging:tag: None 2024-04-26T19:44:11.983 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T19:44:11.983 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:44:11.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T19:44:12.210 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T19:44:12.211 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T19:44:12.218 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T19:44:12.218 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T19:44:12.225 INFO:teuthology.task.internal:Saving configuration 2024-04-26T19:44:12.241 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T19:44:12.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T19:44:12.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:49.632351', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGwX4eQ/wpTKEGvn18+qTf2oYqdCO1H0iRhw2c0Gwmi3Hw2hrx6eYf4A0MKCXS3N6up0unUau9jfXFdbuE8VA0='} 2024-04-26T19:44:12.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:49.633901', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzPW8O5pJzgxy81++2UbljKm0/l8yFx5Z+2tD0wPo7w02Ek+rPl0d2yprJHBgcPk27ea05/YSa6Dt/2oVn9H4E='} 2024-04-26T19:44:12.293 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T19:44:12.299 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T19:44:12.299 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T19:44:12.299 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T19:44:12.364 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c816a6160>, signals=[15]) 2024-04-26T19:44:12.364 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T19:44:12.370 INFO:teuthology.task.internal:Opening connections... 2024-04-26T19:44:12.370 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:44:12.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:44:12.454 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:44:12.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:44:12.528 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T19:44:12.534 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-04-26T19:44:12.551 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-04-26T19:44:12.552 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-26T19:44:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-04-26T19:44:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-04-26T19:44:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-04-26T19:44:12.610 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:44:12.611 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:44:12.612 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-26T19:44:12.636 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-04-26T19:44:12.654 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-04-26T19:44:12.654 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-04-26T19:44:12.711 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-04-26T19:44:12.711 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-04-26T19:44:12.711 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-04-26T19:44:12.711 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:44:12.711 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:44:12.712 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:44:12.712 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-04-26T19:44:12.735 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T19:44:12.744 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T19:44:12.749 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T19:44:12.749 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:44:12.752 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:44:12.767 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T19:44:12.773 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T19:44:12.773 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:44:12.808 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:44:12.828 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T19:44:12.884 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T19:44:12.899 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T19:44:12.900 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T19:44:12.900 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:44:12.901 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:44:12.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:44:12.917 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:12.917 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T19:44:12.917 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-04-26T19:44:12.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:44:12.918 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:12.918 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T19:44:12.918 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-04-26T19:44:12.973 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-437.el9.x86_64 2024-04-26T19:44:12.974 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:12.974 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T19:44:12.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:12.978 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-437.el9.x86_64 2024-04-26T19:44:12.978 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:12.978 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T19:44:12.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:13.193 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum upgrade -y linux-firmware 2024-04-26T19:44:13.200 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum upgrade -y linux-firmware 2024-04-26T19:44:15.404 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T19:44:15.961 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T19:44:17.960 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T19:44:19.663 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-26T19:44:22.127 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T19:44:23.719 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-26T19:44:25.978 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 21 MB 00:02 2024-04-26T19:44:31.061 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T19:44:31.188 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 794 kB/s | 6.2 MB 00:07 2024-04-26T19:44:32.880 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T19:44:33.019 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-04-26T19:44:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-04-26T19:44:33.021 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-04-26T19:44:33.115 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T19:44:33.116 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:33.526 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T19:44:33.526 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T19:44:33.529 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi088.front.sepia.ceph.com to complete... 2024-04-26T19:44:36.010 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:03 2024-04-26T19:44:41.014 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T19:44:43.003 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-26T19:44:43.004 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-04-26T19:44:43.004 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-26T19:44:43.121 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T19:44:43.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T19:44:43.400 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T19:44:43.400 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T19:44:43.403 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi132.front.sepia.ceph.com to complete... 2024-04-26T19:46:48.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26 19:46:48 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T19:46:48.754 INFO:teuthology.orchestra.run.smithi088.stderr:FINISHED --2024-04-26 19:46:48-- 2024-04-26T19:46:48.754 INFO:teuthology.orchestra.run.smithi088.stderr:Total wall clock time: 2m 15s 2024-04-26T19:46:48.754 INFO:teuthology.orchestra.run.smithi088.stderr:Downloaded: 1 files, 341M in 2m 15s (2.53 MB/s) 2024-04-26T19:46:48.758 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T19:46:48.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T19:46:50.862 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying... ######################################## 2024-04-26T19:46:51.091 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing... ######################################## 2024-04-26T19:46:51.209 INFO:teuthology.orchestra.run.smithi088.stdout:Updating / installing... 2024-04-26T19:46:59.144 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T19:47:23.615 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26 19:47:23 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T19:47:23.615 INFO:teuthology.orchestra.run.smithi132.stderr:FINISHED --2024-04-26 19:47:23-- 2024-04-26T19:47:23.616 INFO:teuthology.orchestra.run.smithi132.stderr:Total wall clock time: 2m 40s 2024-04-26T19:47:23.616 INFO:teuthology.orchestra.run.smithi132.stderr:Downloaded: 1 files, 341M in 2m 39s (2.14 MB/s) 2024-04-26T19:47:23.619 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T19:47:23.619 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T19:47:25.733 INFO:teuthology.orchestra.run.smithi132.stdout:Verifying... ######################################## 2024-04-26T19:47:25.862 INFO:teuthology.orchestra.run.smithi132.stdout:Preparing... ######################################## 2024-04-26T19:47:25.977 INFO:teuthology.orchestra.run.smithi132.stdout:Updating / installing... 2024-04-26T19:47:33.885 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T19:47:36.196 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T19:47:36.197 DEBUG:teuthology.orchestra.run.smithi088:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T19:47:36.281 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T19:47:36.282 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T19:47:36.283 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T19:47:36.284 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T19:47:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T19:47:36.286 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T19:47:36.287 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T19:47:36.288 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T19:47:36.289 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T19:47:36.290 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T19:47:36.291 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T19:47:36.292 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T19:47:36.293 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T19:47:36.294 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T19:47:36.295 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T19:47:36.296 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T19:47:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T19:47:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T19:47:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T19:47:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T19:47:36.302 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T19:47:36.303 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T19:47:36.304 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T19:47:36.305 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T19:47:36.306 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T19:47:36.307 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T19:47:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T19:47:36.309 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T19:47:36.310 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T19:47:36.311 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T19:47:36.312 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T19:47:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T19:47:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T19:47:36.315 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T19:47:36.316 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T19:47:36.317 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T19:47:36.318 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T19:47:36.319 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T19:47:36.320 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T19:47:36.321 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T19:47:36.322 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T19:47:36.323 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T19:47:36.324 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T19:47:36.325 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T19:47:36.326 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T19:47:36.327 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T19:47:36.328 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T19:47:36.329 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T19:47:36.330 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T19:47:36.331 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T19:47:36.332 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T19:47:36.333 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T19:47:36.334 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T19:47:36.335 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T19:47:36.336 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T19:47:36.337 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T19:47:36.338 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T19:47:36.339 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T19:47:36.341 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T19:47:36.342 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T19:47:36.343 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T19:47:36.344 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T19:47:36.345 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T19:47:36.346 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T19:47:36.347 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T19:47:36.348 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T19:47:36.349 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T19:47:36.350 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T19:47:36.351 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T19:47:36.352 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T19:47:36.353 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T19:47:36.354 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T19:47:36.355 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T19:47:36.356 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T19:47:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T19:47:36.358 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T19:47:36.359 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T19:47:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T19:47:36.361 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T19:47:36.362 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T19:47:36.363 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T19:47:36.364 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T19:47:36.365 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T19:47:36.367 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T19:47:36.368 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T19:47:36.369 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T19:47:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T19:47:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T19:47:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T19:47:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T19:47:36.374 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T19:47:36.375 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T19:47:36.376 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T19:47:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T19:47:36.378 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T19:47:36.379 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T19:47:36.380 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T19:47:36.381 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T19:47:36.382 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T19:47:36.383 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T19:47:36.384 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T19:47:36.385 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T19:47:36.386 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T19:47:36.387 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T19:47:36.388 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T19:47:36.389 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T19:47:36.390 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T19:47:36.391 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T19:47:36.392 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T19:47:36.393 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T19:47:36.394 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T19:47:36.395 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T19:47:36.396 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T19:47:36.397 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T19:47:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T19:47:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T19:47:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T19:47:36.401 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T19:47:36.402 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T19:47:36.403 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T19:47:36.404 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T19:47:36.405 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T19:47:36.406 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T19:47:36.407 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T19:47:36.408 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T19:47:36.409 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T19:47:36.410 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T19:47:36.411 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T19:47:36.412 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T19:47:36.413 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T19:47:36.414 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T19:47:36.415 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T19:47:36.416 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T19:47:36.417 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T19:47:36.418 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T19:47:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T19:47:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T19:47:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T19:47:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T19:47:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:47:36.423 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.423 DEBUG:teuthology.orchestra.run.smithi088:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T19:47:36.446 INFO:teuthology.orchestra.run.smithi088.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T19:47:36.446 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:47:36.446 INFO:teuthology.orchestra.run.smithi088.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:for file in vmlinuz System.map config; do 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: fi 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:done 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-04-26T19:47:36.447 INFO:teuthology.orchestra.run.smithi088.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T19:47:36.448 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:47:36.448 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.448 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-04-26T19:47:36.449 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2024-04-26T19:47:36.529 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.06 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 68.el9 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Group : Unspecified 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 8264254 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T19:47:36.530 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : builder@centos.org 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:47:36.531 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T19:47:36.532 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T19:47:36.532 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:36.532 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T19:47:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-04-26T19:47:38.524 INFO:teuthology.orchestra.run.smithi088.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T19:47:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-04-26T19:47:38.553 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-04-26T19:47:38.569 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.chelOlaTtT 2024-04-26T19:47:38.569 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.chelOlaTtT 2024-04-26T19:47:38.635 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.chelOlaTtT 2024-04-26T19:47:38.768 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.chelOlaTtT is 6KB 2024-04-26T19:47:38.817 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.chelOlaTtT 2024-04-26T19:47:38.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T19:47:38.897 INFO:teuthology.orchestra.run.smithi088.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T19:47:38.897 INFO:teuthology.orchestra.run.smithi088.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T19:47:38.897 INFO:teuthology.orchestra.run.smithi088.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T19:47:38.897 INFO:teuthology.orchestra.run.smithi088.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T19:47:38.899 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T19:47:39.054 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-04-26T19:48:09.056 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T19:48:09.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:48:09.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:11.996 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T19:48:11.997 DEBUG:teuthology.orchestra.run.smithi132:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T19:48:12.075 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.075 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.075 INFO:teuthology.orchestra.run.smithi132.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.075 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T19:48:12.075 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T19:48:12.076 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T19:48:12.077 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T19:48:12.078 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T19:48:12.079 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T19:48:12.080 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T19:48:12.081 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T19:48:12.082 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T19:48:12.083 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T19:48:12.084 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T19:48:12.085 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T19:48:12.086 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T19:48:12.087 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T19:48:12.088 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T19:48:12.089 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T19:48:12.090 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T19:48:12.091 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T19:48:12.092 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T19:48:12.093 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T19:48:12.094 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T19:48:12.095 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T19:48:12.096 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T19:48:12.097 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T19:48:12.098 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T19:48:12.099 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T19:48:12.100 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T19:48:12.101 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T19:48:12.102 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T19:48:12.103 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T19:48:12.104 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T19:48:12.105 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T19:48:12.106 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T19:48:12.107 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T19:48:12.108 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T19:48:12.109 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T19:48:12.110 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T19:48:12.111 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T19:48:12.112 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T19:48:12.113 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T19:48:12.114 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T19:48:12.115 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T19:48:12.116 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T19:48:12.117 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T19:48:12.118 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T19:48:12.119 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T19:48:12.120 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T19:48:12.121 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T19:48:12.122 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T19:48:12.123 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T19:48:12.124 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T19:48:12.125 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T19:48:12.126 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T19:48:12.127 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T19:48:12.128 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T19:48:12.129 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T19:48:12.130 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T19:48:12.131 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T19:48:12.132 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T19:48:12.133 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T19:48:12.134 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T19:48:12.135 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T19:48:12.136 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T19:48:12.137 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T19:48:12.138 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T19:48:12.139 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T19:48:12.140 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T19:48:12.141 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T19:48:12.142 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T19:48:12.143 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T19:48:12.144 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T19:48:12.145 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T19:48:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T19:48:12.147 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T19:48:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T19:48:12.149 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T19:48:12.150 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T19:48:12.151 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T19:48:12.152 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T19:48:12.153 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T19:48:12.154 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T19:48:12.155 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T19:48:12.156 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T19:48:12.157 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T19:48:12.158 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T19:48:12.159 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T19:48:12.160 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T19:48:12.161 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T19:48:12.162 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T19:48:12.163 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T19:48:12.164 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T19:48:12.165 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T19:48:12.166 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T19:48:12.167 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T19:48:12.168 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T19:48:12.169 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T19:48:12.170 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T19:48:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T19:48:12.172 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T19:48:12.173 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T19:48:12.174 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T19:48:12.175 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T19:48:12.176 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T19:48:12.177 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T19:48:12.178 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T19:48:12.179 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T19:48:12.180 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T19:48:12.181 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T19:48:12.182 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T19:48:12.183 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T19:48:12.184 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T19:48:12.185 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T19:48:12.186 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T19:48:12.187 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T19:48:12.188 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T19:48:12.189 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T19:48:12.190 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T19:48:12.191 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T19:48:12.192 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T19:48:12.193 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T19:48:12.194 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T19:48:12.195 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T19:48:12.196 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T19:48:12.197 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T19:48:12.198 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T19:48:12.199 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T19:48:12.200 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T19:48:12.201 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T19:48:12.202 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T19:48:12.203 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T19:48:12.204 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T19:48:12.205 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T19:48:12.206 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T19:48:12.207 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T19:48:12.208 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T19:48:12.209 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T19:48:12.210 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T19:48:12.211 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T19:48:12.212 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T19:48:12.213 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T19:48:12.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T19:48:12.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T19:48:12.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T19:48:12.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T19:48:12.214 INFO:teuthology.orchestra.run.smithi132.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:48:12.218 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.218 DEBUG:teuthology.orchestra.run.smithi132:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T19:48:12.240 INFO:teuthology.orchestra.run.smithi132.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T19:48:12.240 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:48:12.240 INFO:teuthology.orchestra.run.smithi132.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:for file in vmlinuz System.map config; do 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: fi 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:done 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-26T19:48:12.241 INFO:teuthology.orchestra.run.smithi132.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.242 INFO:teuthology.orchestra.run.smithi132.stdout:fi 2024-04-26T19:48:12.242 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-04-26T19:48:12.311 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T19:48:12.312 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:48:12.313 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T19:48:12.314 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T19:48:12.314 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:12.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T19:48:12.844 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-04-26T19:48:14.257 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T19:48:14.286 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-04-26T19:48:14.288 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-26T19:48:14.319 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.xcUsvoWjmZ 2024-04-26T19:48:14.319 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xcUsvoWjmZ 2024-04-26T19:48:14.383 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.xcUsvoWjmZ 2024-04-26T19:48:14.525 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.xcUsvoWjmZ is 6KB 2024-04-26T19:48:14.574 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.xcUsvoWjmZ 2024-04-26T19:48:14.588 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T19:48:14.650 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T19:48:14.650 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T19:48:14.650 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T19:48:14.650 INFO:teuthology.orchestra.run.smithi132.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T19:48:14.651 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T19:48:14.796 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-04-26T19:48:27.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-26T19:48:36.556 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:48:36.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:39.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-26T19:48:44.802 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T19:48:44.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:48:44.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:51.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:48:51.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:48:54.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-26T19:49:03.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-04-26T19:49:09.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:49:09.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:49:12.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:49:12.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:49:41.677 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-26T19:49:42.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:49:42.282 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T19:49:42.282 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-04-26T19:49:42.321 INFO:teuthology.orchestra.run.smithi088.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T19:49:42.321 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T19:49:42.321 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T19:49:42.322 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T19:49:43.323 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T19:49:43.323 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T19:49:43.451 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2024-04-26T19:49:43.496 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:12.140 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T19:50:24.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:50:24.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:50:24.521 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-04-26T19:50:25.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:50:25.114 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T19:50:25.114 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-04-26T19:50:25.168 INFO:teuthology.orchestra.run.smithi132.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T19:50:25.168 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T19:50:25.169 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T19:50:25.169 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T19:50:26.169 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T19:50:26.170 DEBUG:teuthology.orchestra.run.smithi132:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T19:50:26.299 INFO:teuthology.orchestra.run.smithi132.stdout:ttyS1 2024-04-26T19:50:26.392 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:26.392 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T19:50:26.400 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T19:50:26.400 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:50:26.409 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:50:26.464 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T19:50:26.475 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T19:50:26.481 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T19:50:26.482 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:50:26.490 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:50:26.554 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T19:50:26.561 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T19:50:26.561 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:50:26.566 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:50:26.654 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:50:26.657 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:50:26.685 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:50:26.685 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:50:26.690 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T19:50:26.698 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T19:50:26.698 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:50:26.705 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:50:26.791 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T19:50:26.801 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T19:50:26.801 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:50:26.808 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:50:26.863 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:50:26.950 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:50:27.058 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:50:27.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:50:27.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:50:27.222 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:50:27.299 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:50:27.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:50:27.508 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:50:27.596 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:50:27.679 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:50:27.679 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:50:27.764 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-04-26T19:50:27.772 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-04-26T19:50:27.890 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:50:27.914 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:50:28.377 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T19:50:28.385 INFO:teuthology.task.internal:Starting timer... 2024-04-26T19:50:28.385 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T19:50:28.395 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T19:50:28.403 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T19:50:28.403 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-04-26T19:50:28.522 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-04-26T19:50:28.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-04-26T19:50:28.660 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-04-26T19:50:28.664 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T19:50:28.665 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-04-26T19:50:28.720 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-04-26T19:50:28.721 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-04-26T19:50:28.774 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-04-26T19:50:28.774 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive'} 2024-04-26T19:50:28.775 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:50:28.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:50:28.846 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:50:28.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:50:28.926 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T19:50:28.926 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:50:28.996 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:50:29.074 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T19:50:29.085 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T19:50:29.105 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T19:50:29.106 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T19:55:40.434 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2024-04-26T19:55:40.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:55:40.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:55:40.541 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-26T19:55:40.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-26T19:55:40.701 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:55:40.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:55:40.808 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-04-26T19:55:40.960 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-04-26T19:55:40.960 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T19:55:40.971 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T19:55:40.971 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:55:40.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:55:40.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:55:40.979 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:55:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:55:41.069 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:55:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:55:41.112 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:55:41.167 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-04-26T19:55:41.184 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-04-26T19:55:41.194 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-04-26T19:55:41.215 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-04-26T19:55:41.231 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:55:41.256 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:55:41.269 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:55:41.298 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:55:41.415 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.424 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.442 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.450 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:55:41.452 INFO:teuthology.run_tasks:Running task install... 2024-04-26T19:55:41.460 DEBUG:teuthology.task.install:project ceph 2024-04-26T19:55:41.461 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T19:55:41.461 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T19:55:41.461 INFO:teuthology.task.install:Using flavor: default 2024-04-26T19:55:41.466 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T19:55:41.466 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T19:55:41.467 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:55:41.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:55:41.469 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:55:41.469 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:55:41.611 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:55:41.611 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:55:41.621 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:55:41.621 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:55:41.939 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:55:41.940 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:55:41.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:55:41.993 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:55:41.993 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:55:41.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:55:42.035 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:55:42.035 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:55:42.082 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:55:42.082 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:55:42.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:55:42.209 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:55:42.279 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:55:42.319 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:55:42.354 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-04-26T19:55:42.359 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-04-26T19:55:42.398 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-04-26T19:55:42.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-04-26T19:55:42.678 INFO:teuthology.orchestra.run.smithi132.stdout:48 files removed 2024-04-26T19:55:42.717 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:55:42.723 INFO:teuthology.orchestra.run.smithi088.stdout:48 files removed 2024-04-26T19:55:42.762 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:55:44.173 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 82 kB/s | 84 kB 00:01 2024-04-26T19:55:44.175 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 85 kB/s | 84 kB 00:00 2024-04-26T19:55:45.134 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-26T19:55:45.840 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 7.4 kB/s | 12 kB 00:01 2024-04-26T19:55:46.093 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-26T19:55:46.824 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-04-26T19:55:47.466 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.0 MB 00:01 2024-04-26T19:55:47.714 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T19:55:49.715 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T19:55:51.250 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-04-26T19:55:56.292 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-26T19:55:58.279 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T19:55:59.327 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T19:56:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 884 kB/s | 6.2 MB 00:07 2024-04-26T19:56:02.416 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-26T19:56:04.897 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 21 MB 00:02 2024-04-26T19:56:05.235 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T19:56:07.703 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:56:07.704 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:56:07.725 INFO:teuthology.orchestra.run.smithi088.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:56:07.726 INFO:teuthology.orchestra.run.smithi088.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:56:07.728 INFO:teuthology.orchestra.run.smithi088.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:56:07.729 INFO:teuthology.orchestra.run.smithi088.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:56:07.731 INFO:teuthology.orchestra.run.smithi088.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:56:07.735 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:56:07.736 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:56:07.807 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-04-26T19:56:07.825 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-26T19:56:07.825 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:56:07.826 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:56:07.827 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:56:07.828 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:56:07.829 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:56:07.830 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:56:07.831 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:56:07.832 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:56:07.833 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:56:07.834 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:56:07.835 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:56:07.836 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout:Install 148 Packages 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 4 Packages 2024-04-26T19:56:07.837 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:56:07.838 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 222 M 2024-04-26T19:56:07.838 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-04-26T19:56:09.070 INFO:teuthology.orchestra.run.smithi088.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-26T19:56:10.003 INFO:teuthology.orchestra.run.smithi088.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 954 kB/s | 889 kB 00:00 2024-04-26T19:56:10.187 INFO:teuthology.orchestra.run.smithi088.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 796 kB/s | 145 kB 00:00 2024-04-26T19:56:10.926 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-26T19:56:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-26T19:56:13.287 INFO:teuthology.orchestra.run.smithi088.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-04-26T19:56:13.311 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:56:13.313 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:56:13.333 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:56:13.334 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:56:13.335 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:56:13.337 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:56:13.339 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:56:13.342 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:56:13.344 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:56:13.413 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:56:13.434 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:56:13.435 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:56:13.436 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:56:13.437 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:56:13.438 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:56:13.439 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:56:13.440 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:56:13.441 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:56:13.442 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:56:13.443 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:56:13.444 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout:======================================================================================= 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout:Install 148 Packages 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 4 Packages 2024-04-26T19:56:13.445 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:56:13.446 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 222 M 2024-04-26T19:56:13.446 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-04-26T19:56:14.320 INFO:teuthology.orchestra.run.smithi088.stdout:(6/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 966 kB/s | 5.4 MB 00:05 2024-04-26T19:56:14.661 INFO:teuthology.orchestra.run.smithi132.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-26T19:56:15.519 INFO:teuthology.orchestra.run.smithi132.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-26T19:56:15.645 INFO:teuthology.orchestra.run.smithi132.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.1 MB/s | 145 kB 00:00 2024-04-26T19:56:17.047 INFO:teuthology.orchestra.run.smithi088.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-04-26T19:56:18.328 INFO:teuthology.orchestra.run.smithi132.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 901 kB/s | 2.4 MB 00:02 2024-04-26T19:56:18.620 INFO:teuthology.orchestra.run.smithi132.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-26T19:56:19.803 INFO:teuthology.orchestra.run.smithi132.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-26T19:56:22.220 INFO:teuthology.orchestra.run.smithi132.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-04-26T19:56:23.902 INFO:teuthology.orchestra.run.smithi088.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 11 MB 00:06 2024-04-26T19:56:24.011 INFO:teuthology.orchestra.run.smithi088.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 235 kB/s | 25 kB 00:00 2024-04-26T19:56:25.311 INFO:teuthology.orchestra.run.smithi088.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 21 MB 00:16 2024-04-26T19:56:25.611 INFO:teuthology.orchestra.run.smithi088.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 753 kB/s | 225 kB 00:00 2024-04-26T19:56:25.719 INFO:teuthology.orchestra.run.smithi088.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 250 kB/s | 27 kB 00:00 2024-04-26T19:56:25.836 INFO:teuthology.orchestra.run.smithi088.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 274 kB/s | 32 kB 00:00 2024-04-26T19:56:26.120 INFO:teuthology.orchestra.run.smithi088.stdout:(14/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 17 MB 00:11 2024-04-26T19:56:26.370 INFO:teuthology.orchestra.run.smithi088.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 656 kB/s | 163 kB 00:00 2024-04-26T19:56:26.495 INFO:teuthology.orchestra.run.smithi088.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.1 MB/s | 747 kB 00:00 2024-04-26T19:56:26.520 INFO:teuthology.orchestra.run.smithi088.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 850 kB/s | 127 kB 00:00 2024-04-26T19:56:27.162 INFO:teuthology.orchestra.run.smithi088.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 752 kB/s | 501 kB 00:00 2024-04-26T19:56:27.263 INFO:teuthology.orchestra.run.smithi088.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 456 kB/s | 45 kB 00:00 2024-04-26T19:56:27.446 INFO:teuthology.orchestra.run.smithi088.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 740 kB/s | 135 kB 00:00 2024-04-26T19:56:27.730 INFO:teuthology.orchestra.run.smithi088.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 571 kB/s | 161 kB 00:00 2024-04-26T19:56:28.097 INFO:teuthology.orchestra.run.smithi088.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-26T19:56:28.130 INFO:teuthology.orchestra.run.smithi088.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 804 kB/s | 321 kB 00:00 2024-04-26T19:56:28.357 INFO:teuthology.orchestra.run.smithi088.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 298 kB 00:00 2024-04-26T19:56:28.481 INFO:teuthology.orchestra.run.smithi088.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 284 kB/s | 99 kB 00:00 2024-04-26T19:56:28.515 INFO:teuthology.orchestra.run.smithi088.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 545 kB/s | 85 kB 00:00 2024-04-26T19:56:28.624 INFO:teuthology.orchestra.run.smithi088.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-26T19:56:28.749 INFO:teuthology.orchestra.run.smithi088.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 274 kB/s | 34 kB 00:00 2024-04-26T19:56:28.857 INFO:teuthology.orchestra.run.smithi088.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.3 MB/s | 142 kB 00:00 2024-04-26T19:56:28.953 INFO:teuthology.orchestra.run.smithi132.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 1.4 MB/s | 21 MB 00:14 2024-04-26T19:56:29.136 INFO:teuthology.orchestra.run.smithi132.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 138 kB/s | 25 kB 00:00 2024-04-26T19:56:29.704 INFO:teuthology.orchestra.run.smithi132.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 17 MB 00:09 2024-04-26T19:56:29.995 INFO:teuthology.orchestra.run.smithi132.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 774 kB/s | 225 kB 00:00 2024-04-26T19:56:30.065 INFO:teuthology.orchestra.run.smithi088.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.9 MB/s | 3.5 MB 00:01 2024-04-26T19:56:30.112 INFO:teuthology.orchestra.run.smithi132.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 233 kB/s | 27 kB 00:00 2024-04-26T19:56:30.229 INFO:teuthology.orchestra.run.smithi132.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 274 kB/s | 32 kB 00:00 2024-04-26T19:56:30.646 INFO:teuthology.orchestra.run.smithi132.stdout:(14/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 1.3 MB/s | 11 MB 00:08 2024-04-26T19:56:30.696 INFO:teuthology.orchestra.run.smithi132.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.6 MB/s | 747 kB 00:00 2024-04-26T19:56:30.822 INFO:teuthology.orchestra.run.smithi132.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.0 MB/s | 127 kB 00:00 2024-04-26T19:56:30.955 INFO:teuthology.orchestra.run.smithi132.stdout:(17/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 530 kB/s | 163 kB 00:00 2024-04-26T19:56:31.131 INFO:teuthology.orchestra.run.smithi132.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 1.6 MB/s | 501 kB 00:00 2024-04-26T19:56:31.191 INFO:teuthology.orchestra.run.smithi088.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 1.1 MB/s | 3.1 MB 00:02 2024-04-26T19:56:31.239 INFO:teuthology.orchestra.run.smithi132.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 419 kB/s | 45 kB 00:00 2024-04-26T19:56:31.364 INFO:teuthology.orchestra.run.smithi132.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.1 MB/s | 135 kB 00:00 2024-04-26T19:56:31.449 INFO:teuthology.orchestra.run.smithi088.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 967 kB/s | 249 kB 00:00 2024-04-26T19:56:31.523 INFO:teuthology.orchestra.run.smithi132.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.0 MB/s | 161 kB 00:00 2024-04-26T19:56:31.583 INFO:teuthology.orchestra.run.smithi088.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 372 kB/s | 49 kB 00:00 2024-04-26T19:56:31.708 INFO:teuthology.orchestra.run.smithi088.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 132 kB/s | 16 kB 00:00 2024-04-26T19:56:31.723 INFO:teuthology.orchestra.run.smithi132.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 321 kB 00:00 2024-04-26T19:56:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 4.1 MB/s | 7.4 MB 00:01 2024-04-26T19:56:31.899 INFO:teuthology.orchestra.run.smithi132.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 298 kB 00:00 2024-04-26T19:56:31.975 INFO:teuthology.orchestra.run.smithi088.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.0 MB/s | 273 kB 00:00 2024-04-26T19:56:33.225 INFO:teuthology.orchestra.run.smithi088.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 15 kB/s | 18 kB 00:01 2024-04-26T19:56:33.299 INFO:teuthology.orchestra.run.smithi132.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 71 kB/s | 99 kB 00:01 2024-04-26T19:56:33.441 INFO:teuthology.orchestra.run.smithi132.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 605 kB/s | 85 kB 00:00 2024-04-26T19:56:33.540 INFO:teuthology.orchestra.run.smithi088.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 463 kB/s | 762 kB 00:01 2024-04-26T19:56:34.059 INFO:teuthology.orchestra.run.smithi088.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 96 kB/s | 80 kB 00:00 2024-04-26T19:56:34.866 INFO:teuthology.orchestra.run.smithi132.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 2.2 MB/s | 3.1 MB 00:01 2024-04-26T19:56:34.884 INFO:teuthology.orchestra.run.smithi088.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 237 kB/s | 195 kB 00:00 2024-04-26T19:56:34.983 INFO:teuthology.orchestra.run.smithi132.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-26T19:56:35.075 INFO:teuthology.orchestra.run.smithi132.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 375 kB/s | 34 kB 00:00 2024-04-26T19:56:35.133 INFO:teuthology.orchestra.run.smithi088.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 528 kB/s | 803 kB 00:01 2024-04-26T19:56:35.175 INFO:teuthology.orchestra.run.smithi132.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T19:56:35.976 INFO:teuthology.orchestra.run.smithi088.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 232 kB/s | 253 kB 00:01 2024-04-26T19:56:36.017 INFO:teuthology.orchestra.run.smithi132.stdout:(30/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 1.1 MB/s | 5.4 MB 00:05 2024-04-26T19:56:36.685 INFO:teuthology.orchestra.run.smithi088.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 151 kB/s | 106 kB 00:00 2024-04-26T19:56:37.026 INFO:teuthology.orchestra.run.smithi132.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 1.9 MB/s | 3.5 MB 00:01 2024-04-26T19:56:37.235 INFO:teuthology.orchestra.run.smithi088.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 619 kB/s | 1.2 MB 00:02 2024-04-26T19:56:37.253 INFO:teuthology.orchestra.run.smithi132.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.1 MB/s | 249 kB 00:00 2024-04-26T19:56:37.368 INFO:teuthology.orchestra.run.smithi132.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 433 kB/s | 49 kB 00:00 2024-04-26T19:56:37.410 INFO:teuthology.orchestra.run.smithi088.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 187 kB/s | 135 kB 00:00 2024-04-26T19:56:37.477 INFO:teuthology.orchestra.run.smithi132.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 153 kB/s | 16 kB 00:00 2024-04-26T19:56:37.677 INFO:teuthology.orchestra.run.smithi132.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 273 kB 00:00 2024-04-26T19:56:37.952 INFO:teuthology.orchestra.run.smithi088.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 174 kB/s | 125 kB 00:00 2024-04-26T19:56:38.060 INFO:teuthology.orchestra.run.smithi132.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-26T19:56:38.160 INFO:teuthology.orchestra.run.smithi088.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.3 MB/s | 689 kB 00:00 2024-04-26T19:56:38.169 INFO:teuthology.orchestra.run.smithi132.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 168 kB/s | 18 kB 00:00 2024-04-26T19:56:38.244 INFO:teuthology.orchestra.run.smithi088.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 258 kB/s | 215 kB 00:00 2024-04-26T19:56:38.286 INFO:teuthology.orchestra.run.smithi132.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 694 kB/s | 80 kB 00:00 2024-04-26T19:56:38.308 INFO:teuthology.orchestra.run.smithi088.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.3 MB/s | 943 kB 00:00 2024-04-26T19:56:38.361 INFO:teuthology.orchestra.run.smithi088.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 415 kB/s | 21 kB 00:00 2024-04-26T19:56:38.411 INFO:teuthology.orchestra.run.smithi132.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-26T19:56:38.428 INFO:teuthology.orchestra.run.smithi088.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-26T19:56:38.462 INFO:teuthology.orchestra.run.smithi132.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-26T19:56:38.512 INFO:teuthology.orchestra.run.smithi132.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-26T19:56:38.513 INFO:teuthology.orchestra.run.smithi088.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 400 kB/s | 107 kB 00:00 2024-04-26T19:56:38.596 INFO:teuthology.orchestra.run.smithi088.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-04-26T19:56:38.604 INFO:teuthology.orchestra.run.smithi132.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.2 MB 00:00 2024-04-26T19:56:38.638 INFO:teuthology.orchestra.run.smithi088.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 447 kB/s | 18 kB 00:00 2024-04-26T19:56:38.654 INFO:teuthology.orchestra.run.smithi132.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-26T19:56:38.703 INFO:teuthology.orchestra.run.smithi088.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 172 kB/s | 11 kB 00:00 2024-04-26T19:56:38.713 INFO:teuthology.orchestra.run.smithi132.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T19:56:38.763 INFO:teuthology.orchestra.run.smithi088.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T19:56:38.764 INFO:teuthology.orchestra.run.smithi132.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-26T19:56:38.847 INFO:teuthology.orchestra.run.smithi132.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T19:56:38.855 INFO:teuthology.orchestra.run.smithi088.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.8 MB/s | 3.0 MB 00:00 2024-04-26T19:56:38.872 INFO:teuthology.orchestra.run.smithi088.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-04-26T19:56:38.930 INFO:teuthology.orchestra.run.smithi088.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-26T19:56:38.973 INFO:teuthology.orchestra.run.smithi088.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.8 MB/s | 582 kB 00:00 2024-04-26T19:56:38.998 INFO:teuthology.orchestra.run.smithi088.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-04-26T19:56:39.023 INFO:teuthology.orchestra.run.smithi088.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 957 kB/s | 47 kB 00:00 2024-04-26T19:56:39.057 INFO:teuthology.orchestra.run.smithi088.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-26T19:56:39.107 INFO:teuthology.orchestra.run.smithi088.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T19:56:39.132 INFO:teuthology.orchestra.run.smithi088.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 473 kB/s | 35 kB 00:00 2024-04-26T19:56:39.283 INFO:teuthology.orchestra.run.smithi088.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-26T19:56:39.383 INFO:teuthology.orchestra.run.smithi088.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-26T19:56:39.458 INFO:teuthology.orchestra.run.smithi088.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-26T19:56:39.517 INFO:teuthology.orchestra.run.smithi088.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-26T19:56:39.550 INFO:teuthology.orchestra.run.smithi088.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-26T19:56:39.642 INFO:teuthology.orchestra.run.smithi088.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T19:56:39.692 INFO:teuthology.orchestra.run.smithi088.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-26T19:56:39.801 INFO:teuthology.orchestra.run.smithi088.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-26T19:56:39.860 INFO:teuthology.orchestra.run.smithi088.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T19:56:39.918 INFO:teuthology.orchestra.run.smithi088.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-26T19:56:39.969 INFO:teuthology.orchestra.run.smithi088.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 522 kB/s | 26 kB 00:00 2024-04-26T19:56:40.077 INFO:teuthology.orchestra.run.smithi088.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T19:56:40.103 INFO:teuthology.orchestra.run.smithi088.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-26T19:56:40.136 INFO:teuthology.orchestra.run.smithi088.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T19:56:40.161 INFO:teuthology.orchestra.run.smithi088.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 603 kB/s | 35 kB 00:00 2024-04-26T19:56:40.197 INFO:teuthology.orchestra.run.smithi132.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 511 kB/s | 689 kB 00:01 2024-04-26T19:56:40.239 INFO:teuthology.orchestra.run.smithi088.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 250 kB 00:00 2024-04-26T19:56:40.306 INFO:teuthology.orchestra.run.smithi088.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T19:56:40.356 INFO:teuthology.orchestra.run.smithi088.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T19:56:40.456 INFO:teuthology.orchestra.run.smithi088.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-26T19:56:40.482 INFO:teuthology.orchestra.run.smithi088.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 340 kB/s | 42 kB 00:00 2024-04-26T19:56:40.515 INFO:teuthology.orchestra.run.smithi088.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-26T19:56:40.549 INFO:teuthology.orchestra.run.smithi088.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-26T19:56:40.591 INFO:teuthology.orchestra.run.smithi088.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-26T19:56:40.616 INFO:teuthology.orchestra.run.smithi088.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 774 kB/s | 51 kB 00:00 2024-04-26T19:56:40.641 INFO:teuthology.orchestra.run.smithi088.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-26T19:56:40.708 INFO:teuthology.orchestra.run.smithi088.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 637 kB/s | 42 kB 00:00 2024-04-26T19:56:40.759 INFO:teuthology.orchestra.run.smithi088.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 901 kB/s | 44 kB 00:00 2024-04-26T19:56:40.817 INFO:teuthology.orchestra.run.smithi088.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-26T19:56:40.868 INFO:teuthology.orchestra.run.smithi088.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 732 kB/s | 36 kB 00:00 2024-04-26T19:56:40.926 INFO:teuthology.orchestra.run.smithi088.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 822 kB/s | 47 kB 00:00 2024-04-26T19:56:40.985 INFO:teuthology.orchestra.run.smithi088.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-26T19:56:41.035 INFO:teuthology.orchestra.run.smithi088.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T19:56:41.102 INFO:teuthology.orchestra.run.smithi088.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-26T19:56:41.152 INFO:teuthology.orchestra.run.smithi088.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T19:56:41.202 INFO:teuthology.orchestra.run.smithi088.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 913 kB/s | 45 kB 00:00 2024-04-26T19:56:41.403 INFO:teuthology.orchestra.run.smithi088.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-26T19:56:41.612 INFO:teuthology.orchestra.run.smithi088.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 54 kB/s | 22 kB 00:00 2024-04-26T19:56:41.861 INFO:teuthology.orchestra.run.smithi088.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-26T19:56:41.964 INFO:teuthology.orchestra.run.smithi132.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 534 kB/s | 943 kB 00:01 2024-04-26T19:56:41.987 INFO:teuthology.orchestra.run.smithi088.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-26T19:56:42.037 INFO:teuthology.orchestra.run.smithi088.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 408 kB/s | 19 kB 00:00 2024-04-26T19:56:42.114 INFO:teuthology.orchestra.run.smithi132.stdout:(49/152): ceph-mgr-diskprediction-local-19.0.0- 1.2 MB/s | 7.4 MB 00:06 2024-04-26T19:56:42.163 INFO:teuthology.orchestra.run.smithi088.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-26T19:56:42.213 INFO:teuthology.orchestra.run.smithi088.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 529 kB/s | 26 kB 00:00 2024-04-26T19:56:42.330 INFO:teuthology.orchestra.run.smithi088.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T19:56:42.347 INFO:teuthology.orchestra.run.smithi088.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 365 kB/s | 49 kB 00:00 2024-04-26T19:56:42.381 INFO:teuthology.orchestra.run.smithi088.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-26T19:56:42.414 INFO:teuthology.orchestra.run.smithi088.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-26T19:56:42.465 INFO:teuthology.orchestra.run.smithi088.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-04-26T19:56:42.507 INFO:teuthology.orchestra.run.smithi088.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-26T19:56:42.532 INFO:teuthology.orchestra.run.smithi088.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 659 kB/s | 43 kB 00:00 2024-04-26T19:56:42.566 INFO:teuthology.orchestra.run.smithi088.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 558 kB/s | 32 kB 00:00 2024-04-26T19:56:42.591 INFO:teuthology.orchestra.run.smithi088.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-04-26T19:56:42.642 INFO:teuthology.orchestra.run.smithi088.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-26T19:56:42.691 INFO:teuthology.orchestra.run.smithi088.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T19:56:42.692 INFO:teuthology.orchestra.run.smithi132.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-04-26T19:56:42.725 INFO:teuthology.orchestra.run.smithi088.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-26T19:56:42.751 INFO:teuthology.orchestra.run.smithi088.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 612 kB/s | 35 kB 00:00 2024-04-26T19:56:42.784 INFO:teuthology.orchestra.run.smithi088.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-26T19:56:42.790 INFO:teuthology.orchestra.run.smithi132.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 130 kB/s | 107 kB 00:00 2024-04-26T19:56:42.842 INFO:teuthology.orchestra.run.smithi088.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.5 MB/s | 144 kB 00:00 2024-04-26T19:56:42.910 INFO:teuthology.orchestra.run.smithi088.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-26T19:56:42.935 INFO:teuthology.orchestra.run.smithi088.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-26T19:56:42.960 INFO:teuthology.orchestra.run.smithi088.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-04-26T19:56:42.985 INFO:teuthology.orchestra.run.smithi088.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2024-04-26T19:56:43.011 INFO:teuthology.orchestra.run.smithi088.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-04-26T19:56:43.052 INFO:teuthology.orchestra.run.smithi088.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-26T19:56:43.103 INFO:teuthology.orchestra.run.smithi088.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 944 kB/s | 46 kB 00:00 2024-04-26T19:56:43.154 INFO:teuthology.orchestra.run.smithi088.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-26T19:56:43.178 INFO:teuthology.orchestra.run.smithi088.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-26T19:56:43.221 INFO:teuthology.orchestra.run.smithi088.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 874 kB/s | 58 kB 00:00 2024-04-26T19:56:43.254 INFO:teuthology.orchestra.run.smithi088.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-26T19:56:43.279 INFO:teuthology.orchestra.run.smithi088.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-04-26T19:56:43.313 INFO:teuthology.orchestra.run.smithi088.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T19:56:43.338 INFO:teuthology.orchestra.run.smithi088.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 492 kB/s | 28 kB 00:00 2024-04-26T19:56:43.364 INFO:teuthology.orchestra.run.smithi088.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 620 kB/s | 31 kB 00:00 2024-04-26T19:56:43.414 INFO:teuthology.orchestra.run.smithi088.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-26T19:56:43.424 INFO:teuthology.orchestra.run.smithi132.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-04-26T19:56:43.464 INFO:teuthology.orchestra.run.smithi088.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-26T19:56:43.489 INFO:teuthology.orchestra.run.smithi088.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-04-26T19:56:43.523 INFO:teuthology.orchestra.run.smithi088.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 879 kB/s | 51 kB 00:00 2024-04-26T19:56:43.557 INFO:teuthology.orchestra.run.smithi088.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-26T19:56:43.590 INFO:teuthology.orchestra.run.smithi088.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-26T19:56:43.624 INFO:teuthology.orchestra.run.smithi088.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-26T19:56:43.649 INFO:teuthology.orchestra.run.smithi088.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-04-26T19:56:43.683 INFO:teuthology.orchestra.run.smithi088.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-26T19:56:43.732 INFO:teuthology.orchestra.run.smithi132.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 302 kB/s | 313 kB 00:01 2024-04-26T19:56:44.041 INFO:teuthology.orchestra.run.smithi088.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-04-26T19:56:44.274 INFO:teuthology.orchestra.run.smithi132.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-04-26T19:56:44.791 INFO:teuthology.orchestra.run.smithi132.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-04-26T19:56:45.008 INFO:teuthology.orchestra.run.smithi088.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.6 MB/s | 3.4 MB 00:01 2024-04-26T19:56:45.575 INFO:teuthology.orchestra.run.smithi132.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 205 kB/s | 160 kB 00:00 2024-04-26T19:56:46.092 INFO:teuthology.orchestra.run.smithi088.stdout:(149/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 2.2 MB/s | 49 MB 00:22 2024-04-26T19:56:46.700 INFO:teuthology.orchestra.run.smithi132.stdout:(57/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 2.8 MB/s | 49 MB 00:17 2024-04-26T19:56:46.742 INFO:teuthology.orchestra.run.smithi132.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 39 kB/s | 45 kB 00:01 2024-04-26T19:56:47.225 INFO:teuthology.orchestra.run.smithi088.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 624 kB/s | 1.3 MB 00:02 2024-04-26T19:56:47.692 INFO:teuthology.orchestra.run.smithi132.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 717 kB/s | 3.0 MB 00:04 2024-04-26T19:56:48.010 INFO:teuthology.orchestra.run.smithi132.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 460 kB/s | 582 kB 00:01 2024-04-26T19:56:48.060 INFO:teuthology.orchestra.run.smithi132.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 489 kB/s | 662 kB 00:01 2024-04-26T19:56:48.368 INFO:teuthology.orchestra.run.smithi132.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-04-26T19:56:48.658 INFO:teuthology.orchestra.run.smithi088.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 697 kB/s | 3.1 MB 00:04 2024-04-26T19:56:48.693 INFO:teuthology.orchestra.run.smithi132.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-04-26T19:56:48.985 INFO:teuthology.orchestra.run.smithi132.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 317 kB/s | 292 kB 00:00 2024-04-26T19:56:49.169 INFO:teuthology.orchestra.run.smithi132.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 236 kB/s | 188 kB 00:00 2024-04-26T19:56:49.277 INFO:teuthology.orchestra.run.smithi132.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 60 kB/s | 35 kB 00:00 2024-04-26T19:56:49.425 INFO:teuthology.orchestra.run.smithi088.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 736 kB/s | 2.4 MB 00:03 2024-04-26T19:56:49.438 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:56:49.438 INFO:teuthology.orchestra.run.smithi088.stdout:Total 5.3 MB/s | 222 MB 00:41 2024-04-26T19:56:51.669 INFO:teuthology.orchestra.run.smithi132.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 680 kB/s | 1.6 MB 00:02 2024-04-26T19:56:52.153 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-04-26T19:56:52.395 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-04-26T19:56:52.396 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-04-26T19:56:53.311 INFO:teuthology.orchestra.run.smithi132.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 556 kB/s | 911 kB 00:01 2024-04-26T19:56:54.061 INFO:teuthology.orchestra.run.smithi132.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 178 kB/s | 133 kB 00:00 2024-04-26T19:56:54.828 INFO:teuthology.orchestra.run.smithi132.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 801 kB/s | 4.4 MB 00:05 2024-04-26T19:56:54.938 INFO:teuthology.orchestra.run.smithi132.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 815 kB/s | 4.7 MB 00:05 2024-04-26T19:56:55.125 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-04-26T19:56:55.125 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-04-26T19:56:55.670 INFO:teuthology.orchestra.run.smithi132.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 119 kB/s | 87 kB 00:00 2024-04-26T19:56:56.654 INFO:teuthology.orchestra.run.smithi132.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 567 kB/s | 1.0 MB 00:01 2024-04-26T19:56:57.487 INFO:teuthology.orchestra.run.smithi132.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 293 kB/s | 244 kB 00:00 2024-04-26T19:56:58.371 INFO:teuthology.orchestra.run.smithi132.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 281 kB/s | 248 kB 00:00 2024-04-26T19:56:58.946 INFO:teuthology.orchestra.run.smithi132.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-04-26T19:56:59.496 INFO:teuthology.orchestra.run.smithi132.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-04-26T19:57:00.263 INFO:teuthology.orchestra.run.smithi132.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 225 kB/s | 172 kB 00:00 2024-04-26T19:57:00.880 INFO:teuthology.orchestra.run.smithi132.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-04-26T19:57:01.605 INFO:teuthology.orchestra.run.smithi132.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 785 kB/s | 5.8 MB 00:07 2024-04-26T19:57:01.629 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-04-26T19:57:01.732 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:57:01.809 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:57:01.914 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:57:01.991 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:57:02.023 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:57:02.143 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:57:02.547 INFO:teuthology.orchestra.run.smithi132.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 267 kB/s | 250 kB 00:00 2024-04-26T19:57:02.568 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:57:02.663 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:57:02.842 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:57:02.872 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:57:02.998 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:57:03.043 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:57:03.099 INFO:teuthology.orchestra.run.smithi132.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 822 kB/s | 6.0 MB 00:07 2024-04-26T19:57:03.408 INFO:teuthology.orchestra.run.smithi132.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-04-26T19:57:03.434 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:57:03.497 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:57:03.526 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:57:03.676 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:57:03.710 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:57:03.796 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:57:03.816 INFO:teuthology.orchestra.run.smithi132.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 108 kB/s | 77 kB 00:00 2024-04-26T19:57:03.828 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:57:03.904 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:57:04.042 INFO:teuthology.orchestra.run.smithi132.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 67 kB/s | 42 kB 00:00 2024-04-26T19:57:04.210 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:57:04.271 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:57:04.487 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:57:04.606 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:57:04.645 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:57:04.711 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:57:04.750 INFO:teuthology.orchestra.run.smithi132.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 289 kB/s | 269 kB 00:00 2024-04-26T19:57:04.814 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:57:04.865 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:57:04.867 INFO:teuthology.orchestra.run.smithi132.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 211 kB/s | 159 kB 00:00 2024-04-26T19:57:04.929 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:57:04.979 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:57:05.027 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:57:05.147 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:57:05.226 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:57:05.262 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:57:05.321 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:57:05.426 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:57:05.465 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:57:05.518 INFO:teuthology.orchestra.run.smithi132.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-04-26T19:57:05.535 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:57:05.572 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:57:05.693 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:57:05.709 INFO:teuthology.orchestra.run.smithi132.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 291 kB/s | 279 kB 00:00 2024-04-26T19:57:05.760 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:57:05.871 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:57:06.055 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:57:06.120 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:57:06.260 INFO:teuthology.orchestra.run.smithi132.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 72 kB/s | 54 kB 00:00 2024-04-26T19:57:06.829 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:57:06.885 INFO:teuthology.orchestra.run.smithi132.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-04-26T19:57:06.925 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:57:06.960 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:57:07.004 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:57:07.055 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:57:07.155 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:57:07.431 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:57:07.513 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:57:07.535 INFO:teuthology.orchestra.run.smithi132.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 68 kB/s | 44 kB 00:00 2024-04-26T19:57:07.595 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:57:07.696 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:57:07.727 INFO:teuthology.orchestra.run.smithi132.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 762 kB/s | 5.1 MB 00:06 2024-04-26T19:57:07.737 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:57:07.778 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:57:07.845 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:57:07.893 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:57:07.956 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:57:08.022 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:57:08.217 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:57:08.282 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:57:08.319 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:57:08.376 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:57:08.411 INFO:teuthology.orchestra.run.smithi132.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 53 kB/s | 36 kB 00:00 2024-04-26T19:57:08.503 INFO:teuthology.orchestra.run.smithi132.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 229 kB/s | 221 kB 00:00 2024-04-26T19:57:08.505 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:57:08.544 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:57:08.591 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:57:08.654 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:57:08.684 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:57:08.890 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:57:09.070 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:57:09.087 INFO:teuthology.orchestra.run.smithi132.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-04-26T19:57:09.115 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:57:09.163 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:57:09.216 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:57:09.277 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:57:09.336 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:57:09.377 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:57:09.395 INFO:teuthology.orchestra.run.smithi132.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 342 kB/s | 305 kB 00:00 2024-04-26T19:57:09.424 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:57:09.472 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:57:09.547 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:57:09.595 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:57:09.737 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:57:09.754 INFO:teuthology.orchestra.run.smithi132.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 87 kB/s | 58 kB 00:00 2024-04-26T19:57:09.800 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:57:09.861 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:57:09.933 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:57:10.089 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:57:10.121 INFO:teuthology.orchestra.run.smithi132.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 129 kB/s | 94 kB 00:00 2024-04-26T19:57:10.157 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:57:10.463 INFO:teuthology.orchestra.run.smithi132.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-04-26T19:57:10.788 INFO:teuthology.orchestra.run.smithi132.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-04-26T19:57:11.105 INFO:teuthology.orchestra.run.smithi132.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-04-26T19:57:11.119 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:57:11.173 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:57:11.206 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:57:11.340 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:57:11.432 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:57:11.546 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:57:11.591 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:57:11.825 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:57:11.863 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:57:11.872 INFO:teuthology.orchestra.run.smithi132.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 716 kB/s | 548 kB 00:00 2024-04-26T19:57:11.976 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:57:12.022 INFO:teuthology.orchestra.run.smithi132.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-26T19:57:12.148 INFO:teuthology.orchestra.run.smithi132.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-04-26T19:57:12.431 INFO:teuthology.orchestra.run.smithi132.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 526 kB/s | 863 kB 00:01 2024-04-26T19:57:12.640 INFO:teuthology.orchestra.run.smithi132.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-04-26T19:57:12.765 INFO:teuthology.orchestra.run.smithi132.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 397 kB/s | 49 kB 00:00 2024-04-26T19:57:12.793 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:57:12.849 INFO:teuthology.orchestra.run.smithi132.stdout:(109/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 62 kB/s | 26 kB 00:00 2024-04-26T19:57:12.874 INFO:teuthology.orchestra.run.smithi132.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2024-04-26T19:57:12.936 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:57:13.083 INFO:teuthology.orchestra.run.smithi132.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-04-26T19:57:13.116 INFO:teuthology.orchestra.run.smithi132.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 567 kB/s | 151 kB 00:00 2024-04-26T19:57:13.209 INFO:teuthology.orchestra.run.smithi132.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-26T19:57:13.250 INFO:teuthology.orchestra.run.smithi132.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2024-04-26T19:57:13.326 INFO:teuthology.orchestra.run.smithi132.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-04-26T19:57:13.368 INFO:teuthology.orchestra.run.smithi132.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-04-26T19:57:13.426 INFO:teuthology.orchestra.run.smithi132.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-26T19:57:13.543 INFO:teuthology.orchestra.run.smithi132.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-26T19:57:13.585 INFO:teuthology.orchestra.run.smithi132.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-04-26T19:57:13.661 INFO:teuthology.orchestra.run.smithi132.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 304 kB/s | 35 kB 00:00 2024-04-26T19:57:13.711 INFO:teuthology.orchestra.run.smithi132.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-26T19:57:13.828 INFO:teuthology.orchestra.run.smithi132.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T19:57:13.862 INFO:teuthology.orchestra.run.smithi132.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 964 kB/s | 144 kB 00:00 2024-04-26T19:57:13.920 INFO:teuthology.orchestra.run.smithi132.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-26T19:57:13.978 INFO:teuthology.orchestra.run.smithi132.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-26T19:57:14.037 INFO:teuthology.orchestra.run.smithi132.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-04-26T19:57:14.096 INFO:teuthology.orchestra.run.smithi132.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-26T19:57:14.155 INFO:teuthology.orchestra.run.smithi132.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-04-26T19:57:14.288 INFO:teuthology.orchestra.run.smithi132.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 349 kB/s | 46 kB 00:00 2024-04-26T19:57:14.339 INFO:teuthology.orchestra.run.smithi132.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-26T19:57:14.389 INFO:teuthology.orchestra.run.smithi132.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 794 kB/s | 79 kB 00:00 2024-04-26T19:57:14.456 INFO:teuthology.orchestra.run.smithi132.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2024-04-26T19:57:14.498 INFO:teuthology.orchestra.run.smithi132.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-26T19:57:14.573 INFO:teuthology.orchestra.run.smithi132.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-26T19:57:14.586 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:57:14.599 INFO:teuthology.orchestra.run.smithi132.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-04-26T19:57:14.686 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:57:14.690 INFO:teuthology.orchestra.run.smithi132.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 243 kB/s | 28 kB 00:00 2024-04-26T19:57:14.708 INFO:teuthology.orchestra.run.smithi132.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-26T19:57:14.722 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:57:14.753 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:57:14.800 INFO:teuthology.orchestra.run.smithi132.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T19:57:14.825 INFO:teuthology.orchestra.run.smithi132.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-04-26T19:57:14.900 INFO:teuthology.orchestra.run.smithi132.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 360 kB/s | 36 kB 00:00 2024-04-26T19:57:14.925 INFO:teuthology.orchestra.run.smithi132.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 512 kB/s | 51 kB 00:00 2024-04-26T19:57:15.026 INFO:teuthology.orchestra.run.smithi132.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-26T19:57:15.051 INFO:teuthology.orchestra.run.smithi132.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 719 kB/s | 90 kB 00:00 2024-04-26T19:57:15.124 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:57:15.143 INFO:teuthology.orchestra.run.smithi132.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-26T19:57:15.168 INFO:teuthology.orchestra.run.smithi132.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-04-26T19:57:15.173 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:57:15.252 INFO:teuthology.orchestra.run.smithi132.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T19:57:15.294 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:57:15.329 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:57:15.335 INFO:teuthology.orchestra.run.smithi132.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-26T19:57:15.407 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:57:15.957 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:57:15.992 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:57:16.084 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:57:16.111 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:57:18.277 INFO:teuthology.orchestra.run.smithi132.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 1.1 MB/s | 3.4 MB 00:03 2024-04-26T19:57:18.486 INFO:teuthology.orchestra.run.smithi132.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.6 MB/s | 1.3 MB 00:00 2024-04-26T19:57:18.619 INFO:teuthology.orchestra.run.smithi132.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-26T19:57:18.832 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:18.873 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:18.953 INFO:teuthology.orchestra.run.smithi132.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 889 kB/s | 3.1 MB 00:03 2024-04-26T19:57:19.523 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:19.560 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:57:19.708 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:57:19.709 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:57:19.709 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:57:19.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:19.761 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:57:24.869 INFO:teuthology.orchestra.run.smithi132.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 869 kB/s | 16 MB 00:19 2024-04-26T19:57:24.878 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:57:24.878 INFO:teuthology.orchestra.run.smithi132.stdout:Total 3.1 MB/s | 222 MB 01:11 2024-04-26T19:57:27.646 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-04-26T19:57:27.887 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-04-26T19:57:27.887 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-04-26T19:57:30.549 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-04-26T19:57:30.549 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-04-26T19:57:32.930 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:57:32.930 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-04-26T19:57:32.931 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:33.164 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:57:33.209 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:57:33.210 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:33.210 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:57:33.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:57:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:57:34.903 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:34.903 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:57:34.903 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:57:34.904 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:57:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:57:35.252 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:57:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:57:35.252 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:57:35.253 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:57:35.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:35.734 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:57:35.803 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:57:35.803 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:35.804 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:57:35.804 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:57:35.804 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:57:35.804 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:35.934 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:57:36.011 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:57:36.075 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:57:36.304 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:57:36.373 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:57:36.465 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:57:36.575 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:57:36.759 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:57:36.815 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:57:36.869 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:57:36.890 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-04-26T19:57:36.902 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:57:37.002 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:57:37.079 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:57:37.178 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:57:37.244 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:57:37.308 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:57:37.446 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:57:37.540 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:57:37.598 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:57:37.657 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:57:37.776 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:57:37.807 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:57:37.876 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:57:37.992 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:57:38.187 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:57:38.217 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:57:38.347 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:57:38.394 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:57:38.734 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:57:38.798 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:57:38.839 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:57:39.003 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:57:39.032 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:57:39.128 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:57:39.128 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:57:39.158 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:57:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:57:39.226 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:57:39.506 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:57:39.567 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:57:39.785 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:57:39.905 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:57:39.942 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:57:39.990 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:57:40.093 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:57:40.145 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:57:40.218 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:57:40.265 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:57:40.268 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:57:40.308 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:57:40.317 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:57:40.461 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:57:40.465 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:57:40.541 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:57:40.576 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:57:40.632 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:57:40.638 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:57:40.699 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:57:40.727 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:57:40.762 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:57:40.785 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:57:40.785 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:40.785 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:57:40.785 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:57:40.786 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:57:40.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:40.810 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:57:40.843 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:57:40.854 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:57:40.892 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:57:40.954 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:57:41.013 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:57:41.052 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:57:41.111 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:57:41.158 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:57:41.290 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:57:41.360 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:57:42.041 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:57:42.141 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:57:42.185 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:57:42.222 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:57:42.230 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:57:42.256 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:57:42.271 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:42.330 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:57:42.382 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:57:42.465 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:57:42.466 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:42.466 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:57:42.466 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:42.539 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:57:42.541 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:57:42.636 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:57:42.636 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:42.637 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:57:42.637 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:57:42.637 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:57:42.637 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:42.645 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:57:42.743 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:57:42.837 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:57:42.887 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:57:42.927 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:57:42.995 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:57:43.035 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:57:43.041 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:57:43.097 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:57:43.124 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:57:43.124 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:57:43.124 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:57:43.124 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:57:43.124 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:57:43.125 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:43.156 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:57:43.340 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:57:43.390 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:57:43.457 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:57:43.510 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:57:43.646 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:57:43.686 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:57:43.736 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:57:43.769 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:57:43.802 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:57:44.001 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:57:44.183 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:57:44.485 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:57:44.535 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:57:44.595 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:57:44.683 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:57:44.750 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:57:44.799 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:57:44.852 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:57:44.902 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:57:44.977 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:57:45.022 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:57:45.164 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:57:45.221 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:57:45.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:57:45.342 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:57:45.490 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:57:45.560 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:57:46.511 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:57:46.568 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:57:46.601 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:57:46.722 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:57:46.796 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:57:46.891 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:57:46.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:57:47.156 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:57:47.184 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:57:47.296 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:57:48.115 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:57:48.165 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:57:48.236 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:57:48.273 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:57:48.321 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:57:48.368 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:57:48.417 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:57:48.762 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:57:48.818 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:57:48.846 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:57:49.767 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:57:49.827 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:57:49.855 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:57:49.889 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:57:49.966 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:57:50.009 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:57:50.053 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:57:50.451 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:57:50.495 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:57:50.616 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:57:50.652 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:57:50.709 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:57:51.242 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:57:51.253 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:57:51.281 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:57:51.381 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:57:51.382 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:57:51.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:57:51.482 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:57:51.683 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:57:51.723 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:57:51.723 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:57:51.867 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:57:51.922 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:57:51.922 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:57:52.062 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:57:52.191 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:57:52.494 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:57:52.494 INFO:teuthology.orchestra.run.smithi088.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T19:57:52.494 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:57:54.091 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:54.131 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:54.777 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:57:54.826 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:57:54.978 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:57:54.979 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:57:54.979 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:57:54.979 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:57:55.018 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:57:58.213 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:57:58.214 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:57:58.215 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:57:58.226 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:57:58.227 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:57:58.228 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:57:58.229 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:57:58.230 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:57:58.231 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:57:58.232 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:57:58.233 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:57:58.234 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:57:58.235 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:57:58.236 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:57:58.237 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:57:58.237 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:58:00.692 INFO:teuthology.orchestra.run.smithi088.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.693 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:58:00.694 INFO:teuthology.orchestra.run.smithi088.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:58:00.695 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:58:00.696 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:58:00.697 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:58:00.698 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:58:00.699 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:58:00.700 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:58:00.701 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:58:00.702 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:00.703 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-04-26T19:58:01.270 DEBUG:teuthology.parallel:result is None 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-04-26T19:58:08.455 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:08.691 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:58:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:58:08.739 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:08.739 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:58:08.739 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:10.357 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:58:10.436 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:10.682 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:58:10.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:11.243 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:58:11.306 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:58:11.307 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:11.307 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:58:11.307 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:58:11.307 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:58:11.307 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:11.436 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:58:11.496 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:58:11.570 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:58:11.810 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:58:11.882 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:58:11.968 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:58:12.078 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:58:12.255 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:58:12.310 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:58:12.363 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:58:12.390 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:58:13.014 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:58:13.053 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:58:13.108 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:58:13.217 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:58:13.254 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:58:14.557 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:58:14.597 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:58:15.694 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:58:15.731 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:58:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:58:16.053 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:58:16.100 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:58:16.170 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:58:16.171 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:16.171 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:58:16.171 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:58:16.171 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:58:16.171 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:16.234 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:58:16.270 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:58:16.397 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:58:16.476 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:58:17.490 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:58:17.520 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:58:17.584 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:17.629 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:58:17.722 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:58:17.723 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:17.723 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:58:17.723 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:17.804 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:58:17.908 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:58:17.909 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:17.909 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:58:17.909 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:58:17.909 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:58:17.909 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:18.322 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:58:18.417 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:23.471 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:58:23.525 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:58:23.631 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:58:23.683 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:58:23.731 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:58:24.064 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:58:24.125 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:58:24.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:58:24.974 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:58:25.026 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:58:25.077 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:58:26.376 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:58:26.484 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:58:26.568 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:58:26.799 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:58:26.840 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:58:26.841 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:58:26.964 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:58:27.013 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:58:27.014 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:58:27.162 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:58:27.303 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:58:27.599 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:58:27.599 INFO:teuthology.orchestra.run.smithi132.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T19:58:27.599 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:58:33.403 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:58:33.404 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:58:33.417 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:58:33.417 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:58:33.417 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:58:33.417 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:58:33.417 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:58:33.418 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:58:33.419 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:58:33.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:58:33.421 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:58:33.422 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:58:33.423 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:58:33.424 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:58:33.425 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:58:33.426 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:35.825 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.826 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.827 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.828 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:58:35.829 INFO:teuthology.orchestra.run.smithi132.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:58:35.830 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:58:35.831 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:58:35.832 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:58:35.833 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.834 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:58:35.835 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:58:35.835 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:58:35.835 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:58:35.835 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:58:35.835 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:58:35.836 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:35.837 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-04-26T19:58:36.444 DEBUG:teuthology.parallel:result is None 2024-04-26T19:58:36.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:58:36.586 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:58:36.658 INFO:teuthology.orchestra.run.smithi088.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:58:36.658 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:58:36.659 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:58:36.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:58:36.797 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:58:36.853 INFO:teuthology.orchestra.run.smithi132.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:58:36.853 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:58:36.853 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:58:36.855 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T19:58:36.855 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:36.855 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:58:36.947 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:36.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:58:37.035 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T19:58:37.036 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:37.036 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:58:37.116 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:58:37.229 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:37.229 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:58:37.298 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:58:37.407 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T19:58:37.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:37.407 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:58:37.481 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:58:37.590 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:37.590 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:58:37.657 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:58:37.761 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T19:58:37.761 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:37.761 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:58:37.833 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:58:37.947 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:37.947 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:58:38.015 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:58:38.123 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T19:58:38.206 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T19:58:38.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2024-04-26T19:58:38.213 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /var/log/ceph 2024-04-26T19:58:38.287 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T19:58:38.288 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T19:58:38.292 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T19:58:38.364 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T19:58:38.364 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T19:58:38.370 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T19:58:38.448 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T19:58:38.448 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T19:58:38.448 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T19:58:38.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:38.448 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T19:58:38.493 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T19:58:38.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T19:58:38.796 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:38.797 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T19:58:38.841 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T19:58:39.026 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T19:58:39.094 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T19:58:39.094 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T19:58:39.095 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': None, 'first_in_suite': False, 'job_id': '7675138', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGwX4eQ/wpTKEGvn18+qTf2oYqdCO1H0iRhw2c0Gwmi3Hw2hrx6eYf4A0MKCXS3N6up0unUau9jfXFdbuE8VA0=', 'smithi132.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzPW8O5pJzgxy81++2UbljKm0/l8yFx5Z+2tD0wPo7w02Ek+rPl0d2yprJHBgcPk27ea05/YSa6Dt/2oVn9H4E='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes', 'tasks.cephfs.test_subvolume.TestSubvolume']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T19:58:39.095 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T19:58:39.103 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T19:58:39.153 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T19:58:39.158 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T19:58:39.248 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:39.248 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T19:58:39.304 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T19:58:39.304 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-26 19:57:52.953144225 +0000 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-26 19:55:13.541843999 +0000 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-26 19:55:13.541843999 +0000 2024-04-26T19:58:39.355 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-26 19:55:13.541843999 +0000 2024-04-26T19:58:39.356 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T19:58:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-26T19:58:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-26T19:58:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000382221 s, 1.3 MB/s 2024-04-26T19:58:39.439 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T19:58:39.496 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-04-26T19:58:39.543 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T19:58:39.543 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-26 19:57:52.954144205 +0000 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-26 19:55:14.151831919 +0000 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-26 19:55:14.151831919 +0000 2024-04-26T19:58:39.544 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-26 19:55:14.151831919 +0000 2024-04-26T19:58:39.544 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T19:58:39.620 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-26T19:58:39.621 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-26T19:58:39.621 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000389778 s, 1.3 MB/s 2024-04-26T19:58:39.624 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T19:58:39.680 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-04-26T19:58:39.729 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T19:58:39.729 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-26 19:57:52.955144185 +0000 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-26 19:55:14.745820156 +0000 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-26 19:55:14.745820156 +0000 2024-04-26T19:58:39.730 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-26 19:55:14.745820156 +0000 2024-04-26T19:58:39.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T19:58:39.805 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-26T19:58:39.805 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-26T19:58:39.806 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000333638 s, 1.5 MB/s 2024-04-26T19:58:39.809 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T19:58:39.867 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-04-26T19:58:39.918 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T19:58:39.932 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-26 19:57:52.955144185 +0000 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-26 19:55:15.344808294 +0000 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-26 19:55:15.344808294 +0000 2024-04-26T19:58:39.933 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-26 19:55:15.344808294 +0000 2024-04-26T19:58:39.933 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T19:58:40.016 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-26T19:58:40.016 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-26T19:58:40.016 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00034233 s, 1.5 MB/s 2024-04-26T19:58:40.020 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T19:58:40.074 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:40.074 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:40.074 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T19:58:40.121 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T19:58:40.121 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-26 19:58:28.101422243 +0000 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-26 19:55:09.453617133 +0000 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-26 19:55:09.453617133 +0000 2024-04-26T19:58:40.204 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-26 19:55:09.453617133 +0000 2024-04-26T19:58:40.205 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T19:58:40.314 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-26T19:58:40.315 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-26T19:58:40.315 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00040068 s, 1.3 MB/s 2024-04-26T19:58:40.318 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T19:58:40.360 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-26 19:58:28.101422243 +0000 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-26 19:55:10.032602228 +0000 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-26 19:55:10.032602228 +0000 2024-04-26T19:58:40.449 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-26 19:55:10.032602228 +0000 2024-04-26T19:58:40.450 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T19:58:40.566 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-26T19:58:40.566 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-26T19:58:40.566 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000289876 s, 1.8 MB/s 2024-04-26T19:58:40.569 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T19:58:40.612 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-04-26T19:58:40.689 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T19:58:40.690 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:40.690 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T19:58:40.690 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:40.690 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:40.690 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-26 19:58:28.102422217 +0000 2024-04-26T19:58:40.691 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-26 19:55:10.627586911 +0000 2024-04-26T19:58:40.691 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-26 19:55:10.627586911 +0000 2024-04-26T19:58:40.691 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-26 19:55:10.627586911 +0000 2024-04-26T19:58:40.691 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T19:58:40.797 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-26T19:58:40.797 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-26T19:58:40.797 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000320034 s, 1.6 MB/s 2024-04-26T19:58:40.800 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T19:58:40.845 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-04-26T19:58:40.923 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T19:58:40.923 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-04-26 19:58:28.103422192 +0000 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-04-26 19:55:11.241571106 +0000 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-04-26 19:55:11.241571106 +0000 2024-04-26T19:58:40.924 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-04-26 19:55:11.241571106 +0000 2024-04-26T19:58:40.924 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T19:58:41.026 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-04-26T19:58:41.026 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-04-26T19:58:41.026 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000286435 s, 1.8 MB/s 2024-04-26T19:58:41.029 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T19:58:41.070 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:41.071 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T19:58:41.071 INFO:tasks.ceph:Generating config... 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] pid file = 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T19:58:41.072 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T19:58:41.072 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T19:58:41.073 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T19:58:41.073 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T19:58:41.073 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T19:58:41.073 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T19:58:41.074 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T19:58:41.075 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T19:58:41.075 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T19:58:41.075 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T19:58:41.202 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T19:58:41.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T19:58:41.328 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:58:41.407 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.88'), ('mon.b', '172.21.15.132'), ('mon.c', '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]')] 2024-04-26T19:58:41.408 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.88,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T19:58:41.408 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:41.409 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T19:58:41.459 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.88 --add b 172.21.15.132 --addv c '[v2:172.21.15.132:3301,v1:172.21.15.132:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:41.602 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:41.602 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: generated fsid 100f2996-174e-4373-bd56-473fe683ba7a 2024-04-26T19:58:41.602 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = quincy 2024-04-26T19:58:41.602 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 0 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:fsid 100f2996-174e-4373-bd56-473fe683ba7a 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:last_changed 2024-04-26T19:58:41.602326+0000 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-04-26T19:58:41.602326+0000 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:min_mon_release 17 (quincy) 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:election_strategy: 1 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:1: [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon.b 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:2: [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] mon.c 2024-04-26T19:58:41.603 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T19:58:41.605 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T19:58:41.654 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 100f2996-174e-4373-bd56-473fe683ba7a... 2024-04-26T19:58:41.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T19:58:41.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout:[global] 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout: chdir = "" 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout: auth supported = cephx 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.772 INFO:teuthology.orchestra.run.smithi132.stdout: filestore xattr use omap = true 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: mon clock drift allowed = 1.000 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd crush chooseleaf type = 0 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: auth debug = true 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on old message = true 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: ms die on bug = true 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: mon pg warn max object skew = 0 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.773 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default size = 2 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow primary affinity = true 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd allow pg remap = true 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on legacy crush tunables = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on crush straw calc version zero = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on no sortbitwise = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on osd down out interval zero = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon warn on too few osds = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: mon_allow_pool_size_one = true 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T19:58:41.774 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: osd default data pool replay window = 5 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon allow pool delete = true 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon cluster log file level = debug 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: debug asserts on shutdown = true 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon health detail to clog = false 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: mon host = "172.21.15.88,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: fsid = 100f2996-174e-4373-bd56-473fe683ba7a 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout:[osd] 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: osd journal size = 100 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.775 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub load threshold = 5.0 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd scrub max interval = 600 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd mclock profile = high_recovery_ops 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd recover clone overlap = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd recovery max chunk = 1048576 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug shutdown = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug op order = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug verify stray on activate = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug trim objects = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd open classes on start = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug pg log writeout = true 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.776 INFO:teuthology.orchestra.run.smithi132.stdout: osd deep scrub update digest min age = 30 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: osd map max advance = 10 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: journal zero on create = true 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: filestore ondisk finisher threads = 3 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: filestore apply finisher threads = 3 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bdev debug aio = true 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: osd debug misdirected ops = true 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bdev async discard = True 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bdev enable discard = True 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore allocator = bitmap 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore block size = 96636764160 2024-04-26T19:58:41.777 INFO:teuthology.orchestra.run.smithi132.stdout: bluestore fsck on mount = True 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluefs = 1/20 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug bluestore = 1/20 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug osd = 20 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug rocksdb = 4/10 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd full ratio = 0.9 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd nearfull ratio = 0.8 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: osd failsafe full ratio = 0.95 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: osd objectstore = bluestore 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr] 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-26T19:58:41.778 INFO:teuthology.orchestra.run.smithi132.stdout: debug mgr = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: mgr/telemetry/nag = false 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug finisher = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout:[mon] 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug mon = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug paxos = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: debug auth = 20 2024-04-26T19:58:41.779 INFO:teuthology.orchestra.run.smithi132.stdout: mon data avail warn = 5 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon mgr mkfs grace = 240 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd reporter subtree level = osd 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon osd prime pg temp = true 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.780 INFO:teuthology.orchestra.run.smithi132.stdout: # 1m isn't quite enough 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: mon_down_mkfs_grace = 2m 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: mon_warn_on_filestore_osds = false 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: mon down mkfs grace = 300 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: mon op complaint time = 120 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout:[client] 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: rgw cache enabled = true 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable ops log = true 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: rgw enable usage log = true 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: client mount timeout = 600 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: debug client = 20 2024-04-26T19:58:41.781 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: pid file = "" 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.a] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.b] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mon.c] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout:[mds] 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds = 20 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: debug mds balancer = 20 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: debug ms = 1 2024-04-26T19:58:41.782 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug frag = True 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: mds debug scatterstat = True 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: mds op complaint time = 180 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: mds verify scatter = True 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: osd op complaint time = 180 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: rados mon op timeout = 900 2024-04-26T19:58:41.783 INFO:teuthology.orchestra.run.smithi132.stdout: rados osd op timeout = 900 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.785 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T19:58:41.786 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.787 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.88,172.21.15.132,[v2:172.21.15.132:3301,v1:172.21.15.132:6790]" 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = 100f2996-174e-4373-bd56-473fe683ba7a 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.788 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug trim objects = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2024-04-26T19:58:41.789 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2024-04-26T19:58:41.790 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: osd op complaint time = 180 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2024-04-26T19:58:41.791 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug client = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug finisher = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2024-04-26T19:58:41.792 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2024-04-26T19:58:41.793 INFO:teuthology.orchestra.run.smithi088.stdout: mon down mkfs grace = 300 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: mon op complaint time = 120 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: client mount timeout = 600 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: debug client = 20 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = "" 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: rados mon op timeout = 900 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout: rados osd op timeout = 900 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2024-04-26T19:58:41.794 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout:[mds] 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: debug mds = 20 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: debug mds balancer = 20 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: mds debug frag = True 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: mds debug scatterstat = True 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: mds op complaint time = 180 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: mds verify scatter = True 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: osd op complaint time = 180 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: rados mon op timeout = 900 2024-04-26T19:58:41.795 INFO:teuthology.orchestra.run.smithi088.stdout: rados osd op timeout = 900 2024-04-26T19:58:41.819 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T19:58:41.819 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T19:58:41.947 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T19:58:41.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:41.948 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T19:58:41.997 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:41.997 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T19:58:42.093 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:58:42.093 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:42.093 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T19:58:42.093 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:58:42.214 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:58:42.214 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:42.311 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:58:42.311 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:42.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T19:58:42.311 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:58:42.402 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:58:42.402 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:42.485 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T19:58:42.485 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T19:58:42.485 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T19:58:42.647 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T19:58:42.657 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T19:58:42.794 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T19:58:42.802 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T19:58:42.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T19:58:42.955 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T19:58:42.964 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T19:58:43.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T19:58:43.195 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T19:58:43.202 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T19:58:43.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T19:58:43.407 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T19:58:43.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T19:58:43.492 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T19:58:43.642 INFO:teuthology.orchestra.run.smithi132.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T19:58:43.651 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T19:58:43.721 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T19:58:43.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T19:58:43.841 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T19:58:43.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T19:58:43.990 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T19:58:44.033 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T19:58:44.152 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T19:58:44.190 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T19:58:44.303 INFO:teuthology.orchestra.run.smithi132.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T19:58:44.342 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T19:58:44.343 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi132.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T19:58:44.343 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T19:58:44.434 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:44.435 INFO:tasks.ceph:role: osd.0 2024-04-26T19:58:44.435 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:58:44.435 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T19:58:44.521 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:58:44.522 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:58:44.928 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-26T19:58:44.935 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-26T19:58:44.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T19:58:45.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T19:58:45.187 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T19:58:45.259 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:45.259 INFO:tasks.ceph:role: osd.1 2024-04-26T19:58:45.259 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:58:45.259 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:58:45.342 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:58:45.343 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:58:45.343 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:58:45.700 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-26T19:58:45.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-26T19:58:45.706 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T19:58:45.824 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T19:58:45.901 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T19:58:45.971 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:45.971 INFO:tasks.ceph:role: osd.2 2024-04-26T19:58:45.971 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:58:45.972 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T19:58:46.056 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:58:46.056 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:58:46.056 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:58:46.056 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:58:46.056 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:58:46.057 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:58:46.057 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:58:46.057 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:58:46.057 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:58:46.057 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:58:46.457 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-26T19:58:46.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-26T19:58:46.463 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T19:58:46.628 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T19:58:46.722 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T19:58:46.804 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:58:46.804 INFO:tasks.ceph:role: osd.3 2024-04-26T19:58:46.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T19:58:46.804 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T19:58:46.891 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:58:46.891 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:58:46.891 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:58:46.891 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:58:46.892 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:58:47.300 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-04-26T19:58:47.306 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-04-26T19:58:47.306 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T19:58:47.445 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T19:58:47.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:47.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.672+0000 7fc312220640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T19:58:47.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.673+0000 7fc312220640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T19:58:47.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.673+0000 7fc312220640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:58:47.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.673+0000 7fc312220640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:58:47.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.673+0000 7fc312220640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:58:47.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:47.677+0000 7fc312220640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T19:58:50.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T19:58:50.744 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:50.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.863+0000 7fd15fdfc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T19:58:50.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.864+0000 7fd15fdfc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T19:58:50.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.864+0000 7fd15fdfc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:58:50.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.864+0000 7fd15fdfc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:58:50.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.864+0000 7fd15fdfc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:58:50.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:50.866+0000 7fd15fdfc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T19:58:53.831 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T19:58:53.920 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:54.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.044+0000 7fa9bd301640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T19:58:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.045+0000 7fa9bd301640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T19:58:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.045+0000 7fa9bd301640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:58:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.045+0000 7fa9bd301640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:58:54.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.045+0000 7fa9bd301640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:58:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:54.047+0000 7fa9bd301640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T19:58:57.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T19:58:57.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:58:57.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.250+0000 7faf0fd44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T19:58:57.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.251+0000 7faf0fd44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T19:58:57.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.251+0000 7faf0fd44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:58:57.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.251+0000 7faf0fd44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:58:57.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.251+0000 7faf0fd44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:58:57.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:58:57.253+0000 7faf0fd44640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T19:59:00.197 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T19:59:00.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T19:59:00.370 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:59:00.370 INFO:tasks.ceph:role: osd.4 2024-04-26T19:59:00.370 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:59:00.370 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T19:59:00.458 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:59:00.458 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:59:00.459 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:59:00.872 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-26T19:59:00.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-26T19:59:00.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T19:59:01.026 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T19:59:01.113 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T19:59:01.199 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:59:01.199 INFO:tasks.ceph:role: osd.5 2024-04-26T19:59:01.199 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:59:01.199 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T19:59:01.282 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:59:01.283 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:59:01.637 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-26T19:59:01.642 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-26T19:59:01.642 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T19:59:01.739 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T19:59:01.810 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T19:59:01.873 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:59:01.873 INFO:tasks.ceph:role: osd.6 2024-04-26T19:59:01.874 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:59:01.874 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:59:01.941 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:59:02.325 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-26T19:59:02.331 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-26T19:59:02.331 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T19:59:02.449 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T19:59:02.521 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T19:59:02.582 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:59:02.582 INFO:tasks.ceph:role: osd.7 2024-04-26T19:59:02.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T19:59:02.582 DEBUG:teuthology.orchestra.run.smithi132:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T19:59:02.659 INFO:teuthology.orchestra.run.smithi132.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:59:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:59:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:59:02.659 INFO:teuthology.orchestra.run.smithi132.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:59:02.660 INFO:teuthology.orchestra.run.smithi132.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:59:03.066 INFO:teuthology.orchestra.run.smithi132.stdout:Discarding blocks...Done. 2024-04-26T19:59:03.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi132.front.sepia.ceph.com -o noatime 2024-04-26T19:59:03.073 DEBUG:teuthology.orchestra.run.smithi132:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T19:59:03.235 DEBUG:teuthology.orchestra.run.smithi132:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T19:59:03.330 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:03.478 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.477+0000 7ff5ca00f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T19:59:03.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.478+0000 7ff5ca00f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T19:59:03.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.478+0000 7ff5ca00f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:59:03.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.478+0000 7ff5ca00f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:59:03.479 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.478+0000 7ff5ca00f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:59:03.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:03.482+0000 7ff5ca00f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T19:59:06.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T19:59:06.802 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:06.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.935+0000 7f56a3c86640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T19:59:06.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.936+0000 7f56a3c86640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T19:59:06.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.936+0000 7f56a3c86640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:59:06.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.936+0000 7f56a3c86640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:59:06.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.936+0000 7f56a3c86640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:59:06.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:06.938+0000 7f56a3c86640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T19:59:09.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:59:09.104 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:59:10.175 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T19:59:10.263 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:10.386 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.386+0000 7f5aa2b1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T19:59:10.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.386+0000 7f5aa2b1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T19:59:10.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.386+0000 7f5aa2b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:59:10.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.386+0000 7f5aa2b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:59:10.387 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.386+0000 7f5aa2b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:59:10.389 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:10.389+0000 7f5aa2b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T19:59:13.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T19:59:13.427 DEBUG:teuthology.orchestra.run.smithi132:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:13.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.542+0000 7f466dc43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T19:59:13.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.543+0000 7f466dc43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T19:59:13.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.543+0000 7f466dc43640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:59:13.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.543+0000 7f466dc43640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:59:13.544 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.543+0000 7f466dc43640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:59:13.546 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:13.545+0000 7f466dc43640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T19:59:16.489 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T19:59:16.573 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T19:59:16.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:16.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T19:59:16.662 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:16.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T19:59:16.743 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:16.743 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T19:59:16.818 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:16.818 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T19:59:16.947 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:16.947 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T19:59:17.033 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:17.034 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T19:59:17.120 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:17.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T19:59:17.203 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.204 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T19:59:17.293 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.294 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T19:59:17.379 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T19:59:17.459 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T19:59:17.542 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.542 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T19:59:17.619 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.619 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T19:59:17.700 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.700 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T19:59:17.827 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:17.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T19:59:17.883 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:17.883 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T19:59:17.934 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.934 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T19:59:17.990 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:17.990 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T19:59:18.040 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T19:59:18.040 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T19:59:18.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T19:59:18.124 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2024-04-26T19:59:18.124 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQByByxm0PeZJhAAmWcbnXTpRhnfZRdvKzbTTg== 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQByByxm0jr1OBAAQRkT9BgmjXURkkrr7MAHWQ== 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmCnKlCxAAo0jGLXT6bRL3qpWufJJ5Sw== 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB3ByxmU+QbKBAAKGuevMSCLF8SYrdlt/62YQ== 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB6ByxmJeeNMxAA64NhSfVOpwUK5gJ6cQVDvQ== 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2024-04-26T19:59:18.125 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB+Byxm0QbGAhAA6ufvhdQMCNOPOhioCEovNg== 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCBByxmpToFDxAAAuw5Ew77CrBtkai6YP0AlA== 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQByByxm0kFbLxAApmvQFSMZqcrF7d0MGYbgQw== 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmpzxOGBAADbar8X1eXjXWjfB60egiBQ== 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmRfxVJhAAGgPI5k+fa3OuMZs2kt0msg== 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2024-04-26T19:59:18.126 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCHByxmou+EHBAAkwkBnJbDkuczBKTj/szbYA== 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCKByxmNUjQNxAAz8gNq0pCsDTYh5Fo7xzwbg== 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCOByxmDaIOFxAAoLuo0jYUxlFBf/2VpwDkPw== 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCRByxmaXFiIBAAb4ALfWrAFga124RrQFb4Jw== 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw== 2024-04-26T19:59:18.127 INFO:teuthology.orchestra.run.smithi088.stdout:[client.2] 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmEZsPOxAAJZIw3+ol9NxsWyB851hSRg== 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB0ByxmbJEcCRAA2X7/8G8AR+PRlmM/59Ebqg== 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi088.stdout:[client.3] 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB0Byxm+tkaEhAAXoURy4RGA/OZsj7sUhk8ZQ== 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.y] 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQByByxm0PeZJhAAmWcbnXTpRhnfZRdvKzbTTg== 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.a] 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQByByxm0jr1OBAAQRkT9BgmjXURkkrr7MAHWQ== 2024-04-26T19:59:18.128 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.c] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBzByxmCnKlCxAAo0jGLXT6bRL3qpWufJJ5Sw== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.0] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQB3ByxmU+QbKBAAKGuevMSCLF8SYrdlt/62YQ== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.1] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQB6ByxmJeeNMxAA64NhSfVOpwUK5gJ6cQVDvQ== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.2] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQB+Byxm0QbGAhAA6ufvhdQMCNOPOhioCEovNg== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.3] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCBByxmpToFDxAAAuw5Ew77CrBtkai6YP0AlA== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[mgr.x] 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQByByxm0kFbLxAApmvQFSMZqcrF7d0MGYbgQw== 2024-04-26T19:59:18.129 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.b] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBzByxmpzxOGBAADbar8X1eXjXWjfB60egiBQ== 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout:[mds.d] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBzByxmRfxVJhAAGgPI5k+fa3OuMZs2kt0msg== 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.4] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCHByxmou+EHBAAkwkBnJbDkuczBKTj/szbYA== 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.5] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCKByxmNUjQNxAAz8gNq0pCsDTYh5Fo7xzwbg== 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.6] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCOByxmDaIOFxAAoLuo0jYUxlFBf/2VpwDkPw== 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout:[osd.7] 2024-04-26T19:59:18.130 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCRByxmaXFiIBAAb4ALfWrAFga124RrQFb4Jw== 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout:[client.0] 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw== 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout:[client.2] 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQBzByxmEZsPOxAAJZIw3+ol9NxsWyB851hSRg== 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQB0ByxmbJEcCRAA2X7/8G8AR+PRlmM/59Ebqg== 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout:[client.3] 2024-04-26T19:59:18.131 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQB0Byxm+tkaEhAAXoURy4RGA/OZsj7sUhk8ZQ== 2024-04-26T19:59:18.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:59:18.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:59:18.315 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:18.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:18.452 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:18.459 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:18.596 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:18.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:18.727 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:18.732 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:18.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:18.867 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.000 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.007 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.135 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:59:19.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:59:19.269 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:19.274 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:19.405 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:19.409 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:59:19.548 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.555 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.695 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.828 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.834 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:19.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:59:20.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.101 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.237 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.244 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.377 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.384 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.521 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:59:20.661 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T19:59:20.662 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T19:59:20.744 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:59:20.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T19:59:20.958 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T19:59:21.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:59:21.195 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T19:59:21.272 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T19:59:21.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:59:21.497 DEBUG:teuthology.orchestra.run.smithi132:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T19:59:21.572 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:21.580 DEBUG:teuthology.orchestra.run.smithi132:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:59:21.632 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T19:59:21.633 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T19:59:21.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T19:59:21.638 INFO:tasks.ceph.mon.a:Started 2024-04-26T19:59:21.638 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T19:59:21.638 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T19:59:21.643 INFO:tasks.ceph.mon.b:Started 2024-04-26T19:59:21.643 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T19:59:21.643 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T19:59:21.648 INFO:tasks.ceph.mon.c:Started 2024-04-26T19:59:21.648 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T19:59:21.648 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T19:59:21.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T19:59:21.653 INFO:tasks.ceph.mgr.y:Started 2024-04-26T19:59:21.653 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T19:59:21.653 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T19:59:21.658 INFO:tasks.ceph.mgr.x:Started 2024-04-26T19:59:21.658 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T19:59:21.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T19:59:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.830+0000 7fcff6400640 1 Processor -- start 2024-04-26T19:59:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.831+0000 7fcff6400640 1 -- start start 2024-04-26T19:59:21.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.832+0000 7fcff6400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.832+0000 7fcff6400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 0x7fcff00d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.832+0000 7fcff6400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff0160af0 con 0x7fcff00d0070 2024-04-26T19:59:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.832+0000 7fcff6400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fcff00d3fe0 con 0x7fcff015f2a0 2024-04-26T19:59:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.832+0000 7fcff6400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fcff00d0520 con 0x7fcff00cfc10 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33654/0 (socket says 172.21.15.88:33654) 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 -- 172.21.15.88:0/2948517297 learned_addr learned my addr 172.21.15.88:0/2948517297 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcfefe00640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 0x7fcff00d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 msgr2=0x7fcff0163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T19:59:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 msgr2=0x7fcff0163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcff5000640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcfefe00640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 msgr2=0x7fcff00d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcfefe00640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 msgr2=0x7fcff00d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcfefe00640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 0x7fcff00d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:59:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.833+0000 7fcfefe00640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 0x7fcff00d1a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:59:21.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.846+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1811786725 0 0) 0x7fcff0160af0 con 0x7fcff00d0070 2024-04-26T19:59:21.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.846+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfd8003520 con 0x7fcff00d0070 2024-04-26T19:59:21.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3697045561 0 0) 0x7fcfd8003520 con 0x7fcff00d0070 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 msgr2=0x7fcff00d1a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.132:3300/0 conn(0x7fcff00cfc10 0x7fcff00d1a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 msgr2=0x7fcff0163890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 --2- 172.21.15.88:0/2948517297 >> v2:172.21.15.88:3300/0 conn(0x7fcff015f2a0 0x7fcff0163890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff00d0800 con 0x7fcff00d0070 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fcfe80019f0 con 0x7fcff00d0070 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 >> v1:172.21.15.132:6789/0 conn(0x7fcff00d0070 legacy=0x7fcff0161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfd8003a70 0x7fcfd8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00d3fe0 con 0x7fcfd80051c0 2024-04-26T19:59:21.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00d0520 con 0x7fcfd8004800 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd800c170 con 0x7fcfd8003a70 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcff5a00640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.848+0000 7fcff5000640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfd8003a70 0x7fcfd8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcfefe00640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcff5a00640 1 -- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfd8003a70 msgr2=0x7fcfd8003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcff5a00640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcfd8003a70 0x7fcfd8003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcff5a00640 1 -- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfd8004800 msgr2=0x7fcfd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcff5a00640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfd8004800 0x7fcfd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.849+0000 7fcff5a00640 1 -- 172.21.15.88:0/2948517297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd8003520 con 0x7fcfd80051c0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.850+0000 7fcff5a00640 1 -- 172.21.15.88:0/2948517297 reap_dead start 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.850+0000 7fcff5a00640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c3b0 tx=0x7fcfd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e21b887574dd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.850+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8033070 con 0x7fcfd80051c0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.850+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe802f680 con 0x7fcfd80051c0 2024-04-26T19:59:21.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.850+0000 7fcfef400640 1 -- 172.21.15.88:0/2948517297 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800c0e0 con 0x7fcfd80051c0 2024-04-26T19:59:21.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 -- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 msgr2=0x7fcfd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:21.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c3b0 tx=0x7fcfd8005830 comp rx=0 tx=0).stop 2024-04-26T19:59:21.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 -- 172.21.15.88:0/2948517297 shutdown_connections 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 --2- 172.21.15.88:0/2948517297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcfd800b9c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 -- 172.21.15.88:0/2948517297 >> 172.21.15.88:0/2948517297 conn(0x7fcff009e290 msgr2=0x7fcff00d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 -- 172.21.15.88:0/2948517297 shutdown_connections 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.851+0000 7fcff6400640 1 -- 172.21.15.88:0/2948517297 wait complete. 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.853+0000 7fcff6400640 1 Processor -- start 2024-04-26T19:59:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.853+0000 7fcff6400640 1 -- start start 2024-04-26T19:59:21.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff015a260 0x7fcff015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcff00cd1b0 0x7fcff00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0163890 con 0x7fcfd80051c0 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00d0dc0 con 0x7fcff015a260 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff6400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcff0163a10 con 0x7fcff00cd1b0 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33668/0 (socket says 172.21.15.88:33668) 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff5000640 1 -- 172.21.15.88:0/4097562489 learned_addr learned my addr 172.21.15.88:0/4097562489 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcff5a00640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcff00cd1b0 0x7fcff00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.854+0000 7fcfefe00640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff015a260 0x7fcff015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 -- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcff00cd1b0 msgr2=0x7fcff00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcff00cd1b0 0x7fcff00cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 -- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff015a260 msgr2=0x7fcff015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff015a260 0x7fcff015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 -- 172.21.15.88:0/4097562489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff01f54f0 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff5000640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4011660 tx=0x7fcfe4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4019070 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe400f9c0 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400f040 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.855+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff00cdee0 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.856+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff00ce4a0 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.856+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfe401c070 con 0x7fcfd80051c0 2024-04-26T19:59:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.856+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcfe401e790 con 0x7fcfd80051c0 2024-04-26T19:59:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.857+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff0069ad0 con 0x7fcfd80051c0 2024-04-26T19:59:21.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.860+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcfe400fd90 con 0x7fcfd80051c0 2024-04-26T19:59:21.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.931+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcff00d5440 con 0x7fcfd80051c0 2024-04-26T19:59:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.934+0000 7fcfecc00640 1 -- 172.21.15.88:0/4097562489 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fcfe401e9e0 con 0x7fcfd80051c0 2024-04-26T19:59:21.935 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2024-04-26T19:59:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 msgr2=0x7fcff0153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4011660 tx=0x7fcfe4011b20 comp rx=0 tx=0).stop 2024-04-26T19:59:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 shutdown_connections 2024-04-26T19:59:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcff00cd1b0 0x7fcff00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcff015a260 0x7fcff015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 --2- 172.21.15.88:0/4097562489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfd80051c0 0x7fcff0153e30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.935+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 >> 172.21.15.88:0/4097562489 conn(0x7fcff009e290 msgr2=0x7fcff00683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.936+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 shutdown_connections 2024-04-26T19:59:21.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:21.936+0000 7fcff6400640 1 -- 172.21.15.88:0/4097562489 wait complete. 2024-04-26T19:59:21.953 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T19:59:21.953 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T19:59:21.954 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:21.954 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T19:59:22.022 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:22.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T19:59:22.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:22.138 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T19:59:22.139 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:22.136+0000 7f0172693040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:59:22.150 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:22.150+0000 7fe1dcd2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:59:22.259 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T19:59:22.259 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T19:59:22.328 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:22.327+0000 7f0172693040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:22.361 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:22.361+0000 7fe1dcd2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:22.371 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:22.371 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T19:59:22.394 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:22.394+0000 7f0172693040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:59:22.427 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:22.427+0000 7fe1dcd2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:59:22.448 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:22.448 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T19:59:22.517 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:22.517 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T19:59:22.631 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T19:59:22.631 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T19:59:22.696 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:22.695+0000 7f0172693040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:59:22.736 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:22.736+0000 7fe1dcd2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:59:22.748 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:22.747+0000 7f0172693040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:59:22.755 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new dd3ad240-72d1-436b-b2ff-4a0233e75d9e 0 2024-04-26T19:59:22.794 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:22.793+0000 7fe1dcd2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:59:22.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.935+0000 7fce6b000640 1 Processor -- start 2024-04-26T19:59:22.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.935+0000 7fce6b000640 1 -- start start 2024-04-26T19:59:22.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.936+0000 7fce6b000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fce64163890 0x7fce640d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.936+0000 7fce6b000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce640d3fe0 con 0x7fce6415f2a0 2024-04-26T19:59:22.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.936+0000 7fce6b000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce64160af0 con 0x7fce640d1f50 2024-04-26T19:59:22.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.936+0000 7fce6b000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fce640d0520 con 0x7fce64163890 2024-04-26T19:59:22.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.937+0000 7fce69200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fce64163890 0x7fce640d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.937+0000 7fce69200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fce64163890 0x7fce640d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:51158/0 (socket says 172.21.15.132:51158) 2024-04-26T19:59:22.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.937+0000 7fce69200640 1 -- 172.21.15.132:0/3623780067 learned_addr learned my addr 172.21.15.132:0/3623780067 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:22.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.937+0000 7fce63e00640 1 -- 172.21.15.132:0/3623780067 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385505448 0 0) 0x7fce64160af0 con 0x7fce640d1f50 2024-04-26T19:59:22.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.938+0000 7fce63e00640 1 -- 172.21.15.132:0/3623780067 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce4c003520 con 0x7fce640d1f50 2024-04-26T19:59:22.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.938+0000 7fce69200640 1 -- 172.21.15.132:0/3623780067 >> v1:172.21.15.132:6789/0 conn(0x7fce640d1f50 legacy=0x7fce64161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.938+0000 7fce69200640 1 -- 172.21.15.132:0/3623780067 >> v1:172.21.15.88:6789/0 conn(0x7fce6415f2a0 legacy=0x7fce640cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.938+0000 7fce69200640 1 -- 172.21.15.132:0/3623780067 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce640d0800 con 0x7fce64163890 2024-04-26T19:59:22.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.939+0000 7fce69200640 1 --2- 172.21.15.132:0/3623780067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce640d1a10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce50009870 tx=0x7fce5002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112d21550f1a74d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:22.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.939+0000 7fce63e00640 1 -- 172.21.15.132:0/3623780067 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50037070 con 0x7fce64163890 2024-04-26T19:59:22.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce63e00640 1 -- 172.21.15.132:0/3623780067 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5002ae50 con 0x7fce64163890 2024-04-26T19:59:22.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce63e00640 1 -- 172.21.15.132:0/3623780067 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50032840 con 0x7fce64163890 2024-04-26T19:59:22.948 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 -- 172.21.15.132:0/3623780067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 msgr2=0x7fce640d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 --2- 172.21.15.132:0/3623780067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce640d1a10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fce50009870 tx=0x7fce5002a300 comp rx=0 tx=0).stop 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 -- 172.21.15.132:0/3623780067 shutdown_connections 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 --2- 172.21.15.132:0/3623780067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce640d1a10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 -- 172.21.15.132:0/3623780067 >> 172.21.15.132:0/3623780067 conn(0x7fce6409e290 msgr2=0x7fce640d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 -- 172.21.15.132:0/3623780067 shutdown_connections 2024-04-26T19:59:22.949 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.940+0000 7fce6b000640 1 -- 172.21.15.132:0/3623780067 wait complete. 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.942+0000 7fce6b000640 1 Processor -- start 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.942+0000 7fce6b000640 1 -- start start 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce640d1f50 0x7fce6415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce6415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:22.950 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fce64162530 con 0x7fce64163890 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fce640d0dc0 con 0x7fce640d1f50 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce6b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fce641626b0 con 0x7fce6415f2a0 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce69200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce69c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce640d1f50 0x7fce6415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce69200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56106/0 (socket says 172.21.15.132:56106) 2024-04-26T19:59:22.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.943+0000 7fce69200640 1 -- 172.21.15.132:0/3865979186 learned_addr learned my addr 172.21.15.132:0/3865979186 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 -- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce640d1f50 msgr2=0x7fce6415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce640d1f50 0x7fce6415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 -- 172.21.15.132:0/3865979186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 msgr2=0x7fce6415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce6415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 -- 172.21.15.132:0/3865979186 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce640b88b0 con 0x7fce6415f2a0 2024-04-26T19:59:22.952 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce69200640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce500090b0 tx=0x7fce500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50037050 con 0x7fce6415f2a0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50043070 con 0x7fce6415f2a0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5003e670 con 0x7fce6415f2a0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce640b8b90 con 0x7fce6415f2a0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.944+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce640b9150 con 0x7fce6415f2a0 2024-04-26T19:59:22.953 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.945+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fce5003e810 con 0x7fce6415f2a0 2024-04-26T19:59:22.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.945+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fce500489a0 con 0x7fce6415f2a0 2024-04-26T19:59:22.954 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.946+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce64069ad0 con 0x7fce6415f2a0 2024-04-26T19:59:22.955 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:22.948+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fce50048460 con 0x7fce6415f2a0 2024-04-26T19:59:23.027 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.027+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd3ad240-72d1-436b-b2ff-4a0233e75d9e", "id": 0} v 0) -- 0x7fce64068090 con 0x7fce6415f2a0 2024-04-26T19:59:23.036 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.035+0000 7fce61600640 1 -- 172.21.15.132:0/3865979186 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd3ad240-72d1-436b-b2ff-4a0233e75d9e", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fce50048ba0 con 0x7fce6415f2a0 2024-04-26T19:59:23.036 INFO:teuthology.orchestra.run.smithi132.stdout:0 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 msgr2=0x7fce6415a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fce500090b0 tx=0x7fce500046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 shutdown_connections 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce64163890 0x7fce6415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fce6415f2a0 0x7fce6415a2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 --2- 172.21.15.132:0/3865979186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce640d1f50 0x7fce6415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 >> 172.21.15.132:0/3865979186 conn(0x7fce6409e290 msgr2=0x7fce640b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 shutdown_connections 2024-04-26T19:59:23.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.038+0000 7fce6b000640 1 -- 172.21.15.132:0/3865979186 wait complete. 2024-04-26T19:59:23.054 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 5a25ad95-9feb-4a79-be65-5667b5252117 1 2024-04-26T19:59:23.277 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.276+0000 7f7432400640 1 Processor -- start 2024-04-26T19:59:23.278 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.276+0000 7f7432400640 1 -- start start 2024-04-26T19:59:23.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.279 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f742c0d58b0 0x7f742c0d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f742c0d1d10 0x7f742c0d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0b2070 con 0x7f742c0d58b0 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0d40a0 con 0x7f742c0d1d10 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.277+0000 7f7432400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f742c0d0960 con 0x7f742c0d3780 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f742c0d1d10 0x7f742c0d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56122/0 (socket says 172.21.15.132:56122) 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 -- 172.21.15.132:0/3623294219 learned_addr learned my addr 172.21.15.132:0/3623294219 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742aa00640 1 --2- 172.21.15.132:0/3623294219 >> v2:172.21.15.88:3300/0 conn(0x7f742c0d58b0 0x7f742c0d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 -- 172.21.15.132:0/3623294219 >> v2:172.21.15.132:3300/0 conn(0x7f742c0d1d10 msgr2=0x7f742c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 --2- 172.21.15.132:0/3623294219 >> v2:172.21.15.132:3300/0 conn(0x7f742c0d1d10 0x7f742c0d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 -- 172.21.15.132:0/3623294219 >> v2:172.21.15.88:3300/0 conn(0x7f742c0d58b0 msgr2=0x7f742c0d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:23.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 --2- 172.21.15.132:0/3623294219 >> v2:172.21.15.88:3300/0 conn(0x7f742c0d58b0 0x7f742c0d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.278+0000 7f742b400640 1 -- 172.21.15.132:0/3623294219 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c0d0c40 con 0x7f742c0d3780 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.279+0000 7f742b400640 1 --2- 172.21.15.132:0/3623294219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f742c09e980 tx=0x7f741c02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb22ac34b663e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.279+0000 7f742a000640 1 -- 172.21.15.132:0/3623294219 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c037070 con 0x7f742c0d3780 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.279+0000 7f742a000640 1 -- 172.21.15.132:0/3623294219 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c032440 con 0x7f742c0d3780 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.279+0000 7f742a000640 1 -- 172.21.15.132:0/3623294219 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c032810 con 0x7f742c0d3780 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 -- 172.21.15.132:0/3623294219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 msgr2=0x7f742c0d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 --2- 172.21.15.132:0/3623294219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f742c09e980 tx=0x7f741c02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 -- 172.21.15.132:0/3623294219 shutdown_connections 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 --2- 172.21.15.132:0/3623294219 >> v2:172.21.15.132:3300/0 conn(0x7f742c0d1d10 0x7f742c0d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.282 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 --2- 172.21.15.132:0/3623294219 >> v2:172.21.15.88:3300/0 conn(0x7f742c0d58b0 0x7f742c0d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 --2- 172.21.15.132:0/3623294219 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d3780 0x7f742c0d3b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 -- 172.21.15.132:0/3623294219 >> 172.21.15.132:0/3623294219 conn(0x7f742c09e4e0 msgr2=0x7f742c0d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 -- 172.21.15.132:0/3623294219 shutdown_connections 2024-04-26T19:59:23.283 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.280+0000 7f7432400640 1 -- 172.21.15.132:0/3623294219 wait complete. 2024-04-26T19:59:23.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.282+0000 7f7432400640 1 Processor -- start 2024-04-26T19:59:23.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 -- start start 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f742c0d58b0 0x7f742c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f742c0d32c0 con 0x7f742c0d58b0 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f742c0d3140 con 0x7f742c0d3780 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.283+0000 7f7432400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f742c15f870 con 0x7f742c0d1d10 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36670/0 (socket says 172.21.15.132:36670) 2024-04-26T19:59:23.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56134/0 (socket says 172.21.15.132:56134) 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742aa00640 1 -- 172.21.15.132:0/378862872 learned_addr learned my addr 172.21.15.132:0/378862872 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742be00640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f742c0d58b0 0x7f742c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 -- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 msgr2=0x7f742c0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 -- 172.21.15.132:0/378862872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f742c0d58b0 msgr2=0x7f742c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f742c0d58b0 0x7f742c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 -- 172.21.15.132:0/378862872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c0cdef0 con 0x7f742c0d1d10 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742aa00640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:23.287 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.284+0000 7f742b400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f741c031000 tx=0x7f741c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c041020 con 0x7f742c0d1d10 2024-04-26T19:59:23.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c03d660 con 0x7f742c0d1d10 2024-04-26T19:59:23.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c037040 con 0x7f742c0d1d10 2024-04-26T19:59:23.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c0b47f0 con 0x7f742c0d1d10 2024-04-26T19:59:23.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c0ceaf0 con 0x7f742c0d1d10 2024-04-26T19:59:23.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f741c044020 con 0x7f742c0d1d10 2024-04-26T19:59:23.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.285+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f741c046c10 con 0x7f742c0d1d10 2024-04-26T19:59:23.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.286+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c069650 con 0x7f742c0d1d10 2024-04-26T19:59:23.289 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.288+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f741c04b030 con 0x7f742c0d1d10 2024-04-26T19:59:23.369 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.367+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a25ad95-9feb-4a79-be65-5667b5252117", "id": 1} v 0) -- 0x7f742c0ceef0 con 0x7f742c0d1d10 2024-04-26T19:59:23.377 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.375+0000 7f7423400640 1 -- 172.21.15.132:0/378862872 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a25ad95-9feb-4a79-be65-5667b5252117", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f741c046e10 con 0x7f742c0d1d10 2024-04-26T19:59:23.377 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T19:59:23.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 msgr2=0x7f742c15d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f741c031000 tx=0x7f741c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.378 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 shutdown_connections 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f742c0d58b0 0x7f742c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c0d3780 0x7f742c0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 --2- 172.21.15.132:0/378862872 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f742c0d1d10 0x7f742c15d500 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.376+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 >> 172.21.15.132:0/378862872 conn(0x7f742c09e4e0 msgr2=0x7f742c15fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.377+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 shutdown_connections 2024-04-26T19:59:23.379 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.377+0000 7f7432400640 1 -- 172.21.15.132:0/378862872 wait complete. 2024-04-26T19:59:23.394 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new cfe0e91a-0fc4-49f8-9cc6-109591be2d7f 2 2024-04-26T19:59:23.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.566+0000 7f85d4a00640 1 Processor -- start 2024-04-26T19:59:23.568 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.567+0000 7f85d4a00640 1 -- start start 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85d4a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85d00d19c0 0x7f85d00d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85d4a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d00d3830 con 0x7f85d00d1da0 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85d4a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f85d00d40d0 con 0x7f85d00d19c0 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85d4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00d3b10 con 0x7f85d00b6170 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85cde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85d00d19c0 0x7f85d00d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56144/0 (socket says 172.21.15.132:56144) 2024-04-26T19:59:23.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.568+0000 7f85c7e00640 1 -- 172.21.15.132:0/1222943618 learned_addr learned my addr 172.21.15.132:0/1222943618 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.569+0000 7f85cde00640 1 -- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 msgr2=0x7f85d00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:23.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.569+0000 7f85cde00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.569+0000 7f85cde00640 1 -- 172.21.15.132:0/1222943618 >> v1:172.21.15.132:6789/0 conn(0x7f85d00d1da0 legacy=0x7f85d0162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.569+0000 7f85cde00640 1 -- 172.21.15.132:0/1222943618 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00d3df0 con 0x7f85d00d19c0 2024-04-26T19:59:23.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.569+0000 7f85c7e00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:23.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.570+0000 7f85cde00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d19c0 0x7f85d00d2f00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85c00097a0 tx=0x7f85c002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9dd5ecaa1826596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.570+0000 7f85cd400640 1 -- 172.21.15.132:0/1222943618 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0037070 con 0x7f85d00d19c0 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.570+0000 7f85cd400640 1 -- 172.21.15.132:0/1222943618 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c002ae40 con 0x7f85d00d19c0 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.570+0000 7f85cd400640 1 -- 172.21.15.132:0/1222943618 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0032660 con 0x7f85d00d19c0 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 -- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d19c0 msgr2=0x7f85d00d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d19c0 0x7f85d00d2f00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85c00097a0 tx=0x7f85c002a0f0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 -- 172.21.15.132:0/1222943618 shutdown_connections 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00b6170 0x7f85d00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1222943618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d19c0 0x7f85d00d2f00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 -- 172.21.15.132:0/1222943618 >> 172.21.15.132:0/1222943618 conn(0x7f85d009e460 msgr2=0x7f85d00d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 -- 172.21.15.132:0/1222943618 shutdown_connections 2024-04-26T19:59:23.573 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.571+0000 7f85d4a00640 1 -- 172.21.15.132:0/1222943618 wait complete. 2024-04-26T19:59:23.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.573+0000 7f85d4a00640 1 Processor -- start 2024-04-26T19:59:23.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.573+0000 7f85d4a00640 1 -- start start 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d00b6170 0x7f85d00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d1da0 0x7f85d009fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00cfc30 con 0x7f85d00b6170 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00c91b0 con 0x7f85d00d1da0 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85d4a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f85d0162d50 con 0x7f85d00d19c0 2024-04-26T19:59:23.577 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85ce800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d1da0 0x7f85d009fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56150/0 (socket says 172.21.15.132:56150) 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85c7e00640 1 -- 172.21.15.132:0/1170444758 learned_addr learned my addr 172.21.15.132:0/1170444758 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85cde00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d00b6170 0x7f85d00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85c7e00640 1 -- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d1da0 msgr2=0x7f85d009fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.574+0000 7f85c7e00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d1da0 0x7f85d009fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c7e00640 1 -- 172.21.15.132:0/1170444758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d00b6170 msgr2=0x7f85d00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c7e00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d00b6170 0x7f85d00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c7e00640 1 -- 172.21.15.132:0/1170444758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d00cc290 con 0x7f85d00d19c0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c7e00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85b800e9e0 tx=0x7f85b800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8017070 con 0x7f85d00d19c0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b800c9d0 con 0x7f85d00d19c0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d00cc500 con 0x7f85d00d19c0 2024-04-26T19:59:23.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b800cd80 con 0x7f85d00d19c0 2024-04-26T19:59:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d00a02b0 con 0x7f85d00d19c0 2024-04-26T19:59:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.575+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85b801a070 con 0x7f85d00d19c0 2024-04-26T19:59:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.576+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f85b8006020 con 0x7f85d00d19c0 2024-04-26T19:59:23.579 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.576+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0069650 con 0x7f85d00d19c0 2024-04-26T19:59:23.580 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.579+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f85b80064d0 con 0x7f85d00d19c0 2024-04-26T19:59:23.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.655+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfe0e91a-0fc4-49f8-9cc6-109591be2d7f", "id": 2} v 0) -- 0x7f85d00d26d0 con 0x7f85d00d19c0 2024-04-26T19:59:23.665 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.663+0000 7f85c6000640 1 -- 172.21.15.132:0/1170444758 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfe0e91a-0fc4-49f8-9cc6-109591be2d7f", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f85b8006770 con 0x7f85d00d19c0 2024-04-26T19:59:23.665 INFO:teuthology.orchestra.run.smithi132.stdout:2 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 msgr2=0x7f85d015bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85b800e9e0 tx=0x7f85b800eea0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 shutdown_connections 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d00d1da0 0x7f85d009fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85d00d19c0 0x7f85d015bae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 --2- 172.21.15.132:0/1170444758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d00b6170 0x7f85d00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 >> 172.21.15.132:0/1170444758 conn(0x7f85d009e460 msgr2=0x7f85d006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.667 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 shutdown_connections 2024-04-26T19:59:23.668 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.665+0000 7f85d4a00640 1 -- 172.21.15.132:0/1170444758 wait complete. 2024-04-26T19:59:23.683 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 64bb7329-4c70-4470-9975-7573769e432d 3 2024-04-26T19:59:23.694 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:23.693+0000 7f0172693040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:59:23.723 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:23.722+0000 7fe1dcd2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:59:23.796 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:23.796+0000 7f0172693040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:23.835 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:23.834+0000 7fe1dcd2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:23.846 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:23.846+0000 7f0172693040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:59:23.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.858+0000 7fecdfe00640 1 Processor -- start 2024-04-26T19:59:23.859 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.858+0000 7fecdfe00640 1 -- start start 2024-04-26T19:59:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fece0163a20 0x7fece00d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.862 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fece0162cf0 con 0x7fece0163a20 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fece00d1bb0 con 0x7fece00d1650 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fece0162f40 con 0x7fece015fe40 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdf400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56162/0 (socket says 172.21.15.132:56162) 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdea00640 1 -- 172.21.15.132:0/1658242799 learned_addr learned my addr 172.21.15.132:0/1658242799 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecdf400640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36700/0 (socket says 172.21.15.132:36700) 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.860+0000 7fecde000640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.88:3300/0 conn(0x7fece0163a20 0x7fece00d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdea00640 1 -- 172.21.15.132:0/1658242799 >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 msgr2=0x7fece00b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdea00640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdea00640 1 -- 172.21.15.132:0/1658242799 >> v2:172.21.15.88:3300/0 conn(0x7fece0163a20 msgr2=0x7fece00d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdea00640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.88:3300/0 conn(0x7fece0163a20 0x7fece00d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdea00640 1 -- 172.21.15.132:0/1658242799 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece0163220 con 0x7fece015fe40 2024-04-26T19:59:23.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.861+0000 7fecdf400640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:23.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.862+0000 7fecdea00640 1 --2- 172.21.15.132:0/1658242799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fece00b5850 tx=0x7fecd002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f263f717385bc80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.862+0000 7fecdd600640 1 -- 172.21.15.132:0/1658242799 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0037070 con 0x7fece015fe40 2024-04-26T19:59:23.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.862+0000 7fecdd600640 1 -- 172.21.15.132:0/1658242799 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd0032440 con 0x7fece015fe40 2024-04-26T19:59:23.864 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.862+0000 7fecdd600640 1 -- 172.21.15.132:0/1658242799 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032810 con 0x7fece015fe40 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 -- 172.21.15.132:0/1658242799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 msgr2=0x7fece0160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 --2- 172.21.15.132:0/1658242799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fece00b5850 tx=0x7fecd002a1e0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 -- 172.21.15.132:0/1658242799 shutdown_connections 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.132:3300/0 conn(0x7fece00d1650 0x7fece00b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 --2- 172.21.15.132:0/1658242799 >> v2:172.21.15.88:3300/0 conn(0x7fece0163a20 0x7fece00d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 --2- 172.21.15.132:0/1658242799 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece015fe40 0x7fece0160220 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.865 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 -- 172.21.15.132:0/1658242799 >> 172.21.15.132:0/1658242799 conn(0x7fece0075f90 msgr2=0x7fece006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 -- 172.21.15.132:0/1658242799 shutdown_connections 2024-04-26T19:59:23.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.863+0000 7fecdfe00640 1 -- 172.21.15.132:0/1658242799 wait complete. 2024-04-26T19:59:23.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.865+0000 7fecdfe00640 1 Processor -- start 2024-04-26T19:59:23.866 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.865+0000 7fecdfe00640 1 -- start start 2024-04-26T19:59:23.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece00d1650 0x7fece015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:23.868 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fece00b4b40 con 0x7fece00d1650 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fece00b49c0 con 0x7fece015fe40 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fece00b4cc0 con 0x7fece0163a20 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecde000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdf400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecde000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36710/0 (socket says 172.21.15.132:36710) 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecde000640 1 -- 172.21.15.132:0/4042330431 learned_addr learned my addr 172.21.15.132:0/4042330431 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdf400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56178/0 (socket says 172.21.15.132:56178) 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.866+0000 7fecdea00640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece00d1650 0x7fece015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 -- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 msgr2=0x7fece01f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 -- 172.21.15.132:0/4042330431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece00d1650 msgr2=0x7fece015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece00d1650 0x7fece015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.869 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 -- 172.21.15.132:0/4042330431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece00c7810 con 0x7fece015fe40 2024-04-26T19:59:23.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecdf400640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:23.870 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecde000640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feccc00d800 tx=0x7feccc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00b6c0 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc019070 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00ba80 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece015aee0 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.867+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece015b4a0 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.868+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feccc00bc20 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.868+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7feccc01e8f0 con 0x7fece015fe40 2024-04-26T19:59:23.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.868+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece0069ad0 con 0x7fece015fe40 2024-04-26T19:59:23.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.871+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fece00c7810 con 0x7fece015fe40 2024-04-26T19:59:23.891 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:23.889+0000 7fe1dcd2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:59:23.951 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.950+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "64bb7329-4c70-4470-9975-7573769e432d", "id": 3} v 0) -- 0x7fece00753b0 con 0x7fece015fe40 2024-04-26T19:59:23.959 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.957+0000 7fecd6a00640 1 -- 172.21.15.132:0/4042330431 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64bb7329-4c70-4470-9975-7573769e432d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7feccc01eaf0 con 0x7fece015fe40 2024-04-26T19:59:23.959 INFO:teuthology.orchestra.run.smithi132.stdout:3 2024-04-26T19:59:23.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 msgr2=0x7fece00c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:23.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feccc00d800 tx=0x7feccc00dcc0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 shutdown_connections 2024-04-26T19:59:23.960 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fece0163a20 0x7fece01f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fece015fe40 0x7fece00c72d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 --2- 172.21.15.132:0/4042330431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fece00d1650 0x7fece015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:23.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 >> 172.21.15.132:0/4042330431 conn(0x7fece0075f90 msgr2=0x7fece00b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:23.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 shutdown_connections 2024-04-26T19:59:23.961 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:23.959+0000 7fecdfe00640 1 -- 172.21.15.132:0/4042330431 wait complete. 2024-04-26T19:59:23.977 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 0e15b131-0969-4051-9951-b5b960209504 4 2024-04-26T19:59:24.036 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:24.036+0000 7f0172693040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:59:24.092 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:24.091+0000 7fe1dcd2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:59:24.101 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:24.101+0000 7f0172693040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:59:24.158 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:24.156+0000 7fe1dcd2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:59:24.162 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:24.161+0000 7f0172693040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:59:24.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.197+0000 7fe19dc00640 1 Processor -- start 2024-04-26T19:59:24.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.197+0000 7fe19dc00640 1 -- start start 2024-04-26T19:59:24.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe198163890 0x7fe1980d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe1980d1f50 0x7fe198161e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19dc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe198160af0 con 0x7fe19815f2a0 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19dc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe1980d3fe0 con 0x7fe198163890 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe1980d0520 con 0x7fe1980d1f50 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe197e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fe19815f2a0 0x7fe1980cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:55784/0 (socket says 172.21.15.132:55784) 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe197e00640 1 -- 172.21.15.132:0/2664822939 learned_addr learned my addr 172.21.15.132:0/2664822939 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.200+0000 7fe19d200640 1 --2- 172.21.15.132:0/2664822939 >> v2:172.21.15.132:3300/0 conn(0x7fe1980d1f50 0x7fe198161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe18fe00640 1 --2- 172.21.15.132:0/2664822939 >> v2:172.21.15.88:3300/0 conn(0x7fe198163890 0x7fe1980d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe197400640 1 -- 172.21.15.132:0/2664822939 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361189414 0 0) 0x7fe198160af0 con 0x7fe19815f2a0 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe197400640 1 -- 172.21.15.132:0/2664822939 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe17c003520 con 0x7fe19815f2a0 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe19d200640 1 -- 172.21.15.132:0/2664822939 >> v2:172.21.15.88:3300/0 conn(0x7fe198163890 msgr2=0x7fe1980d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe19d200640 1 --2- 172.21.15.132:0/2664822939 >> v2:172.21.15.88:3300/0 conn(0x7fe198163890 0x7fe1980d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe19d200640 1 -- 172.21.15.132:0/2664822939 >> v1:172.21.15.132:6789/0 conn(0x7fe19815f2a0 legacy=0x7fe1980cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.203 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.201+0000 7fe19d200640 1 -- 172.21.15.132:0/2664822939 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1980d0800 con 0x7fe1980d1f50 2024-04-26T19:59:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.202+0000 7fe19d200640 1 --2- 172.21.15.132:0/2664822939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1980d1f50 0x7fe198161e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe19809e7e0 tx=0x7fe19002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c2fbd5393b684b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.202+0000 7fe197400640 1 -- 172.21.15.132:0/2664822939 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190007740 con 0x7fe1980d1f50 2024-04-26T19:59:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.202+0000 7fe197400640 1 -- 172.21.15.132:0/2664822939 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1900078e0 con 0x7fe1980d1f50 2024-04-26T19:59:24.204 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.202+0000 7fe197400640 1 -- 172.21.15.132:0/2664822939 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190007cb0 con 0x7fe1980d1f50 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 -- 172.21.15.132:0/2664822939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1980d1f50 msgr2=0x7fe198161e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2664822939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1980d1f50 0x7fe198161e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe19809e7e0 tx=0x7fe19002a680 comp rx=0 tx=0).stop 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 -- 172.21.15.132:0/2664822939 shutdown_connections 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2664822939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe1980d1f50 0x7fe198161e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2664822939 >> v2:172.21.15.88:3300/0 conn(0x7fe198163890 0x7fe1980d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 -- 172.21.15.132:0/2664822939 >> 172.21.15.132:0/2664822939 conn(0x7fe19809e290 msgr2=0x7fe1980d5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 -- 172.21.15.132:0/2664822939 shutdown_connections 2024-04-26T19:59:24.205 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.203+0000 7fe19dc00640 1 -- 172.21.15.132:0/2664822939 wait complete. 2024-04-26T19:59:24.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.205+0000 7fe19dc00640 1 Processor -- start 2024-04-26T19:59:24.206 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.205+0000 7fe19dc00640 1 -- start start 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d1f50 0x7fe1980ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe19815f2a0 0x7fe19815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe198162360 con 0x7fe1980d1f50 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980d0dc0 con 0x7fe198163890 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe1981624e0 con 0x7fe19815f2a0 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe18fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe19815f2a0 0x7fe19815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36718/0 (socket says 172.21.15.132:36718) 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe19d200640 1 -- 172.21.15.132:0/2310553169 learned_addr learned my addr 172.21.15.132:0/2310553169 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:24.209 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.206+0000 7fe197e00640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d1f50 0x7fe1980ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 -- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe19815f2a0 msgr2=0x7fe19815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe19815f2a0 0x7fe19815a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 -- 172.21.15.132:0/2310553169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d1f50 msgr2=0x7fe1980ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d1f50 0x7fe1980ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 -- 172.21.15.132:0/2310553169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1980cd1b0 con 0x7fe198163890 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19d200640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe19002ac70 tx=0x7fe190004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190040070 con 0x7fe198163890 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe190031bc0 con 0x7fe198163890 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19815cf10 con 0x7fe198163890 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.207+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190009450 con 0x7fe198163890 2024-04-26T19:59:24.210 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.208+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1980cc190 con 0x7fe198163890 2024-04-26T19:59:24.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.208+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe190039040 con 0x7fe198163890 2024-04-26T19:59:24.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.208+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fe190007840 con 0x7fe198163890 2024-04-26T19:59:24.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.209+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198069ad0 con 0x7fe198163890 2024-04-26T19:59:24.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.211+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe190045d20 con 0x7fe198163890 2024-04-26T19:59:24.235 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:24.233+0000 7fe1dcd2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:59:24.289 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:24.288+0000 7f0172693040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:24.300 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.299+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e15b131-0969-4051-9951-b5b960209504", "id": 4} v 0) -- 0x7fe19808d090 con 0x7fe198163890 2024-04-26T19:59:24.308 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.307+0000 7fe194c00640 1 -- 172.21.15.132:0/2310553169 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e15b131-0969-4051-9951-b5b960209504", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fe190045460 con 0x7fe198163890 2024-04-26T19:59:24.308 INFO:teuthology.orchestra.run.smithi132.stdout:4 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 msgr2=0x7fe19815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe19002ac70 tx=0x7fe190004a40 comp rx=0 tx=0).stop 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 shutdown_connections 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe198163890 0x7fe19815c5a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe19815f2a0 0x7fe19815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 --2- 172.21.15.132:0/2310553169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d1f50 0x7fe1980ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 >> 172.21.15.132:0/2310553169 conn(0x7fe19809e290 msgr2=0x7fe1980cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 shutdown_connections 2024-04-26T19:59:24.310 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.308+0000 7fe19dc00640 1 -- 172.21.15.132:0/2310553169 wait complete. 2024-04-26T19:59:24.325 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66 5 2024-04-26T19:59:24.370 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:24.369+0000 7fe1dcd2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:59:24.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.490+0000 7fcfa1000640 1 Processor -- start 2024-04-26T19:59:24.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.490+0000 7fcfa1000640 1 -- start start 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcfa1000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcf9c0d3c60 0x7fcf9c16a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcfa1000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9c0d0170 con 0x7fcf9c15f7f0 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcfa1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9c0d0450 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcfa1000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9c0d0730 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcf9be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcf9c0d3c60 0x7fcf9c16a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcf9be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fcf9c0d3c60 0x7fcf9c16a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36722/0 (socket says 172.21.15.132:36722) 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcf9be00640 1 -- 172.21.15.132:0/3746480666 learned_addr learned my addr 172.21.15.132:0/3746480666 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:24.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcf9be00640 1 -- 172.21.15.132:0/3746480666 >> v1:172.21.15.132:6789/0 conn(0x7fcf9c0cfb90 legacy=0x7fcf9c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.492+0000 7fcf9be00640 1 -- 172.21.15.132:0/3746480666 >> v1:172.21.15.88:6789/0 conn(0x7fcf9c15f7f0 legacy=0x7fcf9c15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.493+0000 7fcf9be00640 1 -- 172.21.15.132:0/3746480666 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c0d0a10 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.493+0000 7fcf9be00640 1 --2- 172.21.15.132:0/3746480666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c16a060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009e50 tx=0x7fcf9402a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683308403bfad5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcf9a000640 1 -- 172.21.15.132:0/3746480666 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94036070 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcf9a000640 1 -- 172.21.15.132:0/3746480666 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf94031600 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.495 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcf9a000640 1 -- 172.21.15.132:0/3746480666 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf940319d0 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 -- 172.21.15.132:0/3746480666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c0d3c60 msgr2=0x7fcf9c16a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3746480666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c16a060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009e50 tx=0x7fcf9402a6d0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 -- 172.21.15.132:0/3746480666 shutdown_connections 2024-04-26T19:59:24.496 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3746480666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c16a060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 -- 172.21.15.132:0/3746480666 >> 172.21.15.132:0/3746480666 conn(0x7fcf9c0d4f80 msgr2=0x7fcf9c0d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.494+0000 7fcfa1000640 1 -- 172.21.15.132:0/3746480666 shutdown_connections 2024-04-26T19:59:24.497 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.495+0000 7fcfa1000640 1 -- 172.21.15.132:0/3746480666 wait complete. 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.496+0000 7fcfa1000640 1 Processor -- start 2024-04-26T19:59:24.498 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.496+0000 7fcfa1000640 1 -- start start 2024-04-26T19:59:24.499 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c0b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c15f7f0 0x7fcf9c0b9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c15f5b0 con 0x7fcf9c0d3c60 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c15f430 con 0x7fcf9c15f7f0 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcfa1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c16a5a0 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcf9b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56194/0 (socket says 172.21.15.132:56194) 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.497+0000 7fcf9b400640 1 -- 172.21.15.132:0/3523392559 learned_addr learned my addr 172.21.15.132:0/3523392559 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9be00640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c15f7f0 0x7fcf9c0b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9aa00640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c0b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 -- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c15f7f0 msgr2=0x7fcf9c0b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c15f7f0 0x7fcf9c0b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 -- 172.21.15.132:0/3523392559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf9c0d3c60 msgr2=0x7fcf9c0b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.500 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c0b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 -- 172.21.15.132:0/3523392559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c0b9b80 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf9b400640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b420 tx=0x7fcf8c00b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.498+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011830 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c016070 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011c10 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c0b9e60 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c0ba420 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf8c004020 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.499+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fcf8c011db0 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.502 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.500+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0693c0 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.503 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.502+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcf8c020020 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.578 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.577+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66", "id": 5} v 0) -- 0x7fcf9c08ea90 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.586 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.584+0000 7fcf93400640 1 -- 172.21.15.132:0/3523392559 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fcf8c004310 con 0x7fcf9c0cfb90 2024-04-26T19:59:24.586 INFO:teuthology.orchestra.run.smithi132.stdout:5 2024-04-26T19:59:24.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.585+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 msgr2=0x7fcf9c0cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.585+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b420 tx=0x7fcf8c00b8e0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 shutdown_connections 2024-04-26T19:59:24.587 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf9c15f7f0 0x7fcf9c0b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf9c0d3c60 0x7fcf9c0b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 --2- 172.21.15.132:0/3523392559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf9c0cfb90 0x7fcf9c0cb290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 >> 172.21.15.132:0/3523392559 conn(0x7fcf9c0d4f80 msgr2=0x7fcf9c156e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 shutdown_connections 2024-04-26T19:59:24.588 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.586+0000 7fcfa1000640 1 -- 172.21.15.132:0/3523392559 wait complete. 2024-04-26T19:59:24.603 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new c1d094c0-5169-44b9-b13e-2d2f6ece7cb1 6 2024-04-26T19:59:24.740 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:24.740+0000 7f0172693040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:59:24.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.787+0000 7f71b3c00640 1 Processor -- start 2024-04-26T19:59:24.788 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.787+0000 7f71b3c00640 1 -- start start 2024-04-26T19:59:24.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71b3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.790 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71b3c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71ac0d1da0 0x7f71ac162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71b3c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ac0d3830 con 0x7f71ac0d19c0 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71b3c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f71ac0d40d0 con 0x7f71ac0d1da0 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71b3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac0d3b10 con 0x7f71ac0b6170 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.788+0000 7f71abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56210/0 (socket says 172.21.15.132:56210) 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 -- 172.21.15.132:0/1401516642 learned_addr learned my addr 172.21.15.132:0/1401516642 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71b1c00640 1 --2- 172.21.15.132:0/1401516642 >> v2:172.21.15.88:3300/0 conn(0x7f71ac0d1da0 0x7f71ac162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 -- 172.21.15.132:0/1401516642 >> v2:172.21.15.88:3300/0 conn(0x7f71ac0d1da0 msgr2=0x7f71ac162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 --2- 172.21.15.132:0/1401516642 >> v2:172.21.15.88:3300/0 conn(0x7f71ac0d1da0 0x7f71ac162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 -- 172.21.15.132:0/1401516642 >> v1:172.21.15.88:6789/0 conn(0x7f71ac0d19c0 legacy=0x7f71ac0d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:24.791 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.789+0000 7f71abe00640 1 -- 172.21.15.132:0/1401516642 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac0d3df0 con 0x7f71ac0b6170 2024-04-26T19:59:24.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.790+0000 7f71abe00640 1 --2- 172.21.15.132:0/1401516642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7198009870 tx=0x7f719802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50845df5825922a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.790+0000 7f71ab400640 1 -- 172.21.15.132:0/1401516642 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198037070 con 0x7f71ac0b6170 2024-04-26T19:59:24.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.790+0000 7f71ab400640 1 -- 172.21.15.132:0/1401516642 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198032440 con 0x7f71ac0b6170 2024-04-26T19:59:24.792 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.790+0000 7f71ab400640 1 -- 172.21.15.132:0/1401516642 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198032810 con 0x7f71ac0b6170 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 -- 172.21.15.132:0/1401516642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 msgr2=0x7f71ac0c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 --2- 172.21.15.132:0/1401516642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7198009870 tx=0x7f719802a180 comp rx=0 tx=0).stop 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 -- 172.21.15.132:0/1401516642 shutdown_connections 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 --2- 172.21.15.132:0/1401516642 >> v2:172.21.15.88:3300/0 conn(0x7f71ac0d1da0 0x7f71ac162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 --2- 172.21.15.132:0/1401516642 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0b6170 0x7f71ac0c8c70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 -- 172.21.15.132:0/1401516642 >> 172.21.15.132:0/1401516642 conn(0x7f71ac09e460 msgr2=0x7f71ac0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 -- 172.21.15.132:0/1401516642 shutdown_connections 2024-04-26T19:59:24.793 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.791+0000 7f71b3c00640 1 -- 172.21.15.132:0/1401516642 wait complete. 2024-04-26T19:59:24.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.793+0000 7f71b3c00640 1 Processor -- start 2024-04-26T19:59:24.794 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.793+0000 7f71b3c00640 1 -- start start 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ac0d19c0 0x7f71ac0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 0x7f71ac09ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac162b80 con 0x7f71ac0d19c0 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0cfc30 con 0x7f71ac0b6170 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac162d00 con 0x7f71ac0d1da0 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.796 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 0x7f71ac09ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:36740/0 (socket says 172.21.15.132:36740) 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1200640 1 -- 172.21.15.132:0/2280808948 learned_addr learned my addr 172.21.15.132:0/2280808948 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71abe00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ac0d19c0 0x7f71ac0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 -- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 msgr2=0x7f71ac1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 -- 172.21.15.132:0/2280808948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ac0d19c0 msgr2=0x7f71ac0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ac0d19c0 0x7f71ac0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1c00640 1 -- 172.21.15.132:0/2280808948 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac0ccb60 con 0x7f71ac0d1da0 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.794+0000 7f71b1200640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:24.797 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71b1c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 0x7f71ac09ffc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007cf0 tx=0x7f71a400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:24.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400edd0 con 0x7f71ac0d1da0 2024-04-26T19:59:24.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a4003990 con 0x7f71ac0d1da0 2024-04-26T19:59:24.798 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003d80 con 0x7f71ac0d1da0 2024-04-26T19:59:24.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac0cce40 con 0x7f71ac0d1da0 2024-04-26T19:59:24.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac0cd400 con 0x7f71ac0d1da0 2024-04-26T19:59:24.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71a401b020 con 0x7f71ac0d1da0 2024-04-26T19:59:24.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.795+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f71a4014420 con 0x7f71ac0d1da0 2024-04-26T19:59:24.799 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.796+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac069650 con 0x7f71ac0d1da0 2024-04-26T19:59:24.800 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.798+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f71a4014a60 con 0x7f71ac0d1da0 2024-04-26T19:59:24.853 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:24.852+0000 7fe1dcd2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:59:24.881 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.880+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c1d094c0-5169-44b9-b13e-2d2f6ece7cb1", "id": 6} v 0) -- 0x7f71ac0bafc0 con 0x7f71ac0d1da0 2024-04-26T19:59:24.892 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.891+0000 7f71a8c00640 1 -- 172.21.15.132:0/2280808948 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1d094c0-5169-44b9-b13e-2d2f6ece7cb1", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f71a4014c40 con 0x7f71ac0d1da0 2024-04-26T19:59:24.892 INFO:teuthology.orchestra.run.smithi132.stdout:6 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 msgr2=0x7f71ac09ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 0x7f71ac09ffc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007cf0 tx=0x7f71a400e4b0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 shutdown_connections 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ac0d1da0 0x7f71ac09ffc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ac0d19c0 0x7f71ac0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 --2- 172.21.15.132:0/2280808948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71ac0b6170 0x7f71ac1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 >> 172.21.15.132:0/2280808948 conn(0x7f71ac09e460 msgr2=0x7f71ac161820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 shutdown_connections 2024-04-26T19:59:24.894 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:24.892+0000 7f71b3c00640 1 -- 172.21.15.132:0/2280808948 wait complete. 2024-04-26T19:59:24.911 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --cluster ceph osd new 44bc038e-9483-482f-ac71-acf9b8b70d85 7 2024-04-26T19:59:25.035 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.034+0000 7f0172693040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:59:25.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.083+0000 7f409a800640 1 Processor -- start 2024-04-26T19:59:25.085 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.083+0000 7f409a800640 1 -- start start 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f409a800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f409415f7f0 0x7f40940cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f409a800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40940d0170 con 0x7f40940d2c50 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f409a800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40940d0450 con 0x7f409415fbd0 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f409a800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f40940d0730 con 0x7f409415f7f0 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f4098a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f409415fbd0 0x7f40940d22d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.132:55806/0 (socket says 172.21.15.132:55806) 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f4098a00640 1 -- 172.21.15.132:0/190894551 learned_addr learned my addr 172.21.15.132:0/190894551 (peer_addr_for_me v1:172.21.15.132:0/0) 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f4099400640 1 --2- 172.21.15.132:0/190894551 >> v2:172.21.15.88:3300/0 conn(0x7f409415f7f0 0x7f40940cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268584971 0 0) 0x7f40940d0450 con 0x7f409415fbd0 2024-04-26T19:59:25.087 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.085+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f407c003520 con 0x7f409415fbd0 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2376141941 0 0) 0x7f407c003520 con 0x7f409415fbd0 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 >> v2:172.21.15.88:3300/0 conn(0x7f409415f7f0 msgr2=0x7f40940cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 --2- 172.21.15.132:0/190894551 >> v2:172.21.15.88:3300/0 conn(0x7f409415f7f0 0x7f40940cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 >> v1:172.21.15.88:6789/0 conn(0x7f40940d2c50 legacy=0x7f40940d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 --> v1:172.21.15.132:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40940d0a10 con 0x7f409415fbd0 2024-04-26T19:59:25.091 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v1:172.21.15.132:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f4080001670 con 0x7f409415fbd0 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 >> v1:172.21.15.132:6789/0 conn(0x7f409415fbd0 legacy=0x7f40940d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c003a70 0x7f407c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 0x7f407c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f407c0052e0 0x7f407c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40940d0730 con 0x7f407c0052e0 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f407c00c1b0 con 0x7f407c004800 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f407c00c3b0 con 0x7f407c003a70 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4098a00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 0x7f407c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4099400640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c003a70 0x7f407c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.086+0000 7f4099e00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f407c0052e0 0x7f407c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4098a00640 1 -- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c003a70 msgr2=0x7f407c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4098a00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c003a70 0x7f407c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.092 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4098a00640 1 -- 172.21.15.132:0/190894551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f407c0052e0 msgr2=0x7f407c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4098a00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f407c0052e0 0x7f407c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4098a00640 1 -- 172.21.15.132:0/190894551 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c003520 con 0x7f407c004800 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4099e00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f407c0052e0 0x7f407c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.087+0000 7f4099e00640 1 -- 172.21.15.132:0/190894551 reap_dead start 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f4098a00640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 0x7f407c004c80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f408000c110 tx=0x7f407c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404c20c90ac99da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080007c40 con 0x7f407c004800 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4080005af0 con 0x7f407c004800 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f4093e00640 1 -- 172.21.15.132:0/190894551 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080007070 con 0x7f407c004800 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f409a800640 1 -- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 msgr2=0x7f407c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.088+0000 7f409a800640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 0x7f407c004c80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f408000c110 tx=0x7f407c005830 comp rx=0 tx=0).stop 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.089+0000 7f409a800640 1 -- 172.21.15.132:0/190894551 shutdown_connections 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.089+0000 7f409a800640 1 --2- 172.21.15.132:0/190894551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f407c004800 0x7f407c004c80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.089+0000 7f409a800640 1 -- 172.21.15.132:0/190894551 >> 172.21.15.132:0/190894551 conn(0x7f40940d4f80 msgr2=0x7f40940d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.089+0000 7f409a800640 1 -- 172.21.15.132:0/190894551 shutdown_connections 2024-04-26T19:59:25.093 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.089+0000 7f409a800640 1 -- 172.21.15.132:0/190894551 wait complete. 2024-04-26T19:59:25.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.090+0000 7f409a800640 1 Processor -- start 2024-04-26T19:59:25.095 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.090+0000 7f409a800640 1 -- start start 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40940cde90 0x7f409415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40940cceb0 0x7f40940cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f409415f430 con 0x7f40940cceb0 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4094160050 con 0x7f40940cde90 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f409a800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f409415f5b0 con 0x7f407c004800 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.091+0000 7f4099400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4098a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40940cde90 0x7f409415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.132:56234/0 (socket says 172.21.15.132:56234) 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 -- 172.21.15.132:0/290246794 learned_addr learned my addr 172.21.15.132:0/290246794 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099e00640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40940cceb0 0x7f40940cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.096 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 -- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40940cde90 msgr2=0x7f409415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40940cde90 0x7f409415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 -- 172.21.15.132:0/290246794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40940cceb0 msgr2=0x7f40940cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40940cceb0 0x7f40940cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 -- 172.21.15.132:0/290246794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40940ce600 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4099400640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40880118f0 tx=0x7f4088011db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088019070 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4088004240 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.092+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40940cc0f0 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.093+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408800f7c0 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.093+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40940cc6b0 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.093+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f408801c070 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.093+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4088003cf0 con 0x7f407c004800 2024-04-26T19:59:25.097 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.094+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409415e9f0 con 0x7f407c004800 2024-04-26T19:59:25.098 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.094+0000 7f0172693040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:59:25.098 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.096+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f409415e9f0 con 0x7f407c004800 2024-04-26T19:59:25.163 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.162+0000 7fe1dcd2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:59:25.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.170+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd new", "uuid": "44bc038e-9483-482f-ac71-acf9b8b70d85", "id": 7} v 0) -- 0x7f40940693c0 con 0x7f407c004800 2024-04-26T19:59:25.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.180+0000 7f4091600640 1 -- 172.21.15.132:0/290246794 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "44bc038e-9483-482f-ac71-acf9b8b70d85", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f40880036e0 con 0x7f407c004800 2024-04-26T19:59:25.182 INFO:teuthology.orchestra.run.smithi132.stdout:7 2024-04-26T19:59:25.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 msgr2=0x7f40941f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40880118f0 tx=0x7f4088011db0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 shutdown_connections 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40940cceb0 0x7f40940cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40940cde90 0x7f409415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 --2- 172.21.15.132:0/290246794 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f407c004800 0x7f40941f63f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 >> 172.21.15.132:0/290246794 conn(0x7f40940d4f80 msgr2=0x7f4094160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 shutdown_connections 2024-04-26T19:59:25.184 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T19:59:25.182+0000 7f409a800640 1 -- 172.21.15.132:0/290246794 wait complete. 2024-04-26T19:59:25.199 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T19:59:25.199 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T19:59:25.206 INFO:tasks.ceph.osd.0:Started 2024-04-26T19:59:25.206 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T19:59:25.207 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T19:59:25.215 INFO:tasks.ceph.osd.1:Started 2024-04-26T19:59:25.215 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T19:59:25.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T19:59:25.221 INFO:tasks.ceph.osd.2:Started 2024-04-26T19:59:25.221 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T19:59:25.221 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T19:59:25.227 INFO:tasks.ceph.osd.3:Started 2024-04-26T19:59:25.227 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T19:59:25.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T19:59:25.227 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.226+0000 7fe1dcd2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:59:25.234 INFO:tasks.ceph.osd.4:Started 2024-04-26T19:59:25.234 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T19:59:25.234 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T19:59:25.235 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.235+0000 7f0172693040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:59:25.241 INFO:tasks.ceph.osd.5:Started 2024-04-26T19:59:25.241 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T19:59:25.241 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T19:59:25.248 INFO:tasks.ceph.osd.6:Started 2024-04-26T19:59:25.248 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T19:59:25.248 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T19:59:25.254 INFO:tasks.ceph.osd.7:Started 2024-04-26T19:59:25.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:25.399 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.398+0000 7fe1dcd2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:59:25.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.478+0000 7f7242600640 1 Processor -- start 2024-04-26T19:59:25.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.478+0000 7f7242600640 1 -- start start 2024-04-26T19:59:25.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7242600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f723c0d3690 0x7f723c0d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7242600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c0b4510 0x7f723c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7242600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f723c0cfd60 con 0x7f723c15f450 2024-04-26T19:59:25.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7242600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f723c0d0040 con 0x7f723c0d3690 2024-04-26T19:59:25.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7242600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0d0320 con 0x7f723c0b4510 2024-04-26T19:59:25.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7233e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f723c0d3690 0x7f723c0d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7233e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f723c0d3690 0x7f723c0d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43516/0 (socket says 172.21.15.88:43516) 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f7233e00640 1 -- 172.21.15.88:0/3819102841 learned_addr learned my addr 172.21.15.88:0/3819102841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.479+0000 7f723be00640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c0b4510 0x7f723c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.480+0000 7f7233e00640 1 -- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c0b4510 msgr2=0x7f723c0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.480+0000 7f7233e00640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c0b4510 0x7f723c0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.480+0000 7f7233e00640 1 -- 172.21.15.88:0/3819102841 >> v1:172.21.15.132:6789/0 conn(0x7f723c15f450 legacy=0x7f723c0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.480+0000 7f7233e00640 1 -- 172.21.15.88:0/3819102841 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0d0600 con 0x7f723c0d3690 2024-04-26T19:59:25.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.481+0000 7f7233e00640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0d3690 0x7f723c0d3b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592cf41bc7ea462a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:25.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.481+0000 7f723aa00640 1 -- 172.21.15.88:0/3819102841 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224037070 con 0x7f723c0d3690 2024-04-26T19:59:25.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.481+0000 7f723aa00640 1 -- 172.21.15.88:0/3819102841 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7224032440 con 0x7f723c0d3690 2024-04-26T19:59:25.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.481+0000 7f723aa00640 1 -- 172.21.15.88:0/3819102841 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224032810 con 0x7f723c0d3690 2024-04-26T19:59:25.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 -- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0d3690 msgr2=0x7f723c0d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0d3690 0x7f723c0d3b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7224009870 tx=0x7f722402a250 comp rx=0 tx=0).stop 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 -- 172.21.15.88:0/3819102841 shutdown_connections 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c0b4510 0x7f723c0b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 --2- 172.21.15.88:0/3819102841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0d3690 0x7f723c0d3b20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 -- 172.21.15.88:0/3819102841 >> 172.21.15.88:0/3819102841 conn(0x7f723c0d50b0 msgr2=0x7f723c0d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 -- 172.21.15.88:0/3819102841 shutdown_connections 2024-04-26T19:59:25.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.482+0000 7f7242600640 1 -- 172.21.15.88:0/3819102841 wait complete. 2024-04-26T19:59:25.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.483+0000 7f7242600640 1 Processor -- start 2024-04-26T19:59:25.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.484+0000 7f7242600640 1 -- start start 2024-04-26T19:59:25.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.484+0000 7f7242600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 0x7f723c0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.484+0000 7f7242600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.484+0000 7f7242600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c15f450 0x7f723c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:25.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7242600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f723c0d5e00 con 0x7f723c0d3690 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7242600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f723c15f830 con 0x7f723c0b4510 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7242600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f723c0d0e00 con 0x7f723c15f450 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33696/0 (socket says 172.21.15.88:33696) 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 -- 172.21.15.88:0/2149692124 learned_addr learned my addr 172.21.15.88:0/2149692124 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f723be00640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c15f450 0x7f723c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f723b400640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 0x7f723c0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 -- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c15f450 msgr2=0x7f723c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c15f450 0x7f723c1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 -- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 msgr2=0x7f723c0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 0x7f723c0cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 -- 172.21.15.88:0/2149692124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c0ba260 con 0x7f723c0d3690 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f723b400640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 0x7f723c0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:59:25.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.485+0000 7f7233e00640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f72240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224037040 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7224042070 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c0ba500 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224030880 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c0baac0 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7224030a20 con 0x7f723c0d3690 2024-04-26T19:59:25.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.486+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7224030d10 con 0x7f723c0d3690 2024-04-26T19:59:25.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.487+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c069540 con 0x7f723c0d3690 2024-04-26T19:59:25.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.490+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7224047460 con 0x7f723c0d3690 2024-04-26T19:59:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.556+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f723c0d3470 con 0x7f723c0d3690 2024-04-26T19:59:25.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.556+0000 7f7233400640 1 -- 172.21.15.88:0/2149692124 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7224047640 con 0x7f723c0d3690 2024-04-26T19:59:25.557 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:25.557 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":10,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:25.173989+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 msgr2=0x7f723c0b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7224000c00 tx=0x7f72240046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 shutdown_connections 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f723c15f450 0x7f723c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f723c0d3690 0x7f723c0b7c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 --2- 172.21.15.88:0/2149692124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f723c0b4510 0x7f723c0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 >> 172.21.15.88:0/2149692124 conn(0x7f723c0d50b0 msgr2=0x7f723c0b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 shutdown_connections 2024-04-26T19:59:25.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:25.558+0000 7f7242600640 1 -- 172.21.15.88:0/2149692124 wait complete. 2024-04-26T19:59:25.575 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T19:59:25.575 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T19:59:25.625 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.625+0000 7f0172693040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:59:25.682 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.682+0000 7f0172693040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:59:25.745 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.743+0000 7fe1dcd2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:59:25.800 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.799+0000 7fe1dcd2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:59:25.864 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:25.864+0000 7f0172693040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:59:25.902 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-26T19:59:25.901+0000 7ff26f9dc640 -1 Falling back to public interface 2024-04-26T19:59:25.907 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-26T19:59:25.906+0000 7f8fefa1c640 -1 Falling back to public interface 2024-04-26T19:59:25.923 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-26T19:59:25.922+0000 7f91ba916640 -1 Falling back to public interface 2024-04-26T19:59:25.927 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-26T19:59:25.927+0000 7f8b27f2c640 -1 Falling back to public interface 2024-04-26T19:59:25.933 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-26T19:59:25.933+0000 7ff3b7921640 -1 Falling back to public interface 2024-04-26T19:59:25.938 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-26T19:59:25.937+0000 7f18a0220640 -1 Falling back to public interface 2024-04-26T19:59:25.955 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-26T19:59:25.953+0000 7fddec60c640 -1 Falling back to public interface 2024-04-26T19:59:25.955 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-26T19:59:25.954+0000 7f3b94c17640 -1 Falling back to public interface 2024-04-26T19:59:25.991 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:25.989+0000 7fe1dcd2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:59:26.036 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.036+0000 7f0172693040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:59:26.094 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.093+0000 7f0172693040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:59:26.155 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:26.153+0000 7fe1dcd2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:59:26.173 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.173+0000 7f0172693040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:26.214 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:26.212+0000 7fe1dcd2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:59:26.227 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.227+0000 7f0172693040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:59:26.298 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:26.296+0000 7fe1dcd2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:59:26.351 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:26.349+0000 7fe1dcd2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:59:26.379 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:26.534 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.534+0000 7f816b600640 1 Processor -- start 2024-04-26T19:59:26.534 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.534+0000 7f816b600640 1 -- start start 2024-04-26T19:59:26.535 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f816b600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f816415be30 0x7f81640ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:26.535 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f816b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:26.535 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f816b600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81640aecf0 con 0x7f816406feb0 2024-04-26T19:59:26.535 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f816b600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8164070320 con 0x7f816415be30 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f816b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8164072020 con 0x7f816415c210 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f8163e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f8163e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:51566/0 (socket says 172.21.15.88:51566) 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f8163e00640 1 -- 172.21.15.88:0/582011499 learned_addr learned my addr 172.21.15.88:0/582011499 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.535+0000 7f8168c00640 1 --2- 172.21.15.88:0/582011499 >> v2:172.21.15.132:3300/0 conn(0x7f816415be30 0x7f81640ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:26.536 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.536+0000 7f8163e00640 1 -- 172.21.15.88:0/582011499 >> v2:172.21.15.132:3300/0 conn(0x7f816415be30 msgr2=0x7f81640ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:26.537 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.536+0000 7f8163e00640 1 --2- 172.21.15.88:0/582011499 >> v2:172.21.15.132:3300/0 conn(0x7f816415be30 0x7f81640ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.537 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.536+0000 7f8163e00640 1 -- 172.21.15.88:0/582011499 >> v1:172.21.15.132:6789/0 conn(0x7f816406feb0 legacy=0x7f81640b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:26.537 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.536+0000 7f8163e00640 1 -- 172.21.15.88:0/582011499 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640b4ae0 con 0x7f816415c210 2024-04-26T19:59:26.537 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.537+0000 7f8163e00640 1 --2- 172.21.15.88:0/582011499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f815000bb10 tx=0x7f815002c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61258ee54cfc2916 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:26.538 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.537+0000 7f8163400640 1 -- 172.21.15.88:0/582011499 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150036020 con 0x7f816415c210 2024-04-26T19:59:26.538 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.537+0000 7f8163400640 1 -- 172.21.15.88:0/582011499 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150032820 con 0x7f816415c210 2024-04-26T19:59:26.538 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.537+0000 7f8163400640 1 -- 172.21.15.88:0/582011499 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150032bf0 con 0x7f816415c210 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 -- 172.21.15.88:0/582011499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 msgr2=0x7f816406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 --2- 172.21.15.88:0/582011499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f815000bb10 tx=0x7f815002c920 comp rx=0 tx=0).stop 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 -- 172.21.15.88:0/582011499 shutdown_connections 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 --2- 172.21.15.88:0/582011499 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816415c210 0x7f816406f970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 --2- 172.21.15.88:0/582011499 >> v2:172.21.15.132:3300/0 conn(0x7f816415be30 0x7f81640ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 -- 172.21.15.88:0/582011499 >> 172.21.15.88:0/582011499 conn(0x7f81640d0c40 msgr2=0x7f81640d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 -- 172.21.15.88:0/582011499 shutdown_connections 2024-04-26T19:59:26.539 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.538+0000 7f816b600640 1 -- 172.21.15.88:0/582011499 wait complete. 2024-04-26T19:59:26.540 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.540+0000 7f816b600640 1 Processor -- start 2024-04-26T19:59:26.540 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.540+0000 7f816b600640 1 -- start start 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816406feb0 0x7f8164155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 0x7f8164154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81640728a0 con 0x7f816415be30 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8164072600 con 0x7f816415c210 2024-04-26T19:59:26.542 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f816b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f81640b4480 con 0x7f816406feb0 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33704/0 (socket says 172.21.15.88:33704) 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 -- 172.21.15.88:0/4152958511 learned_addr learned my addr 172.21.15.88:0/4152958511 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8168c00640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816406feb0 0x7f8164155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8169600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 0x7f8164154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 -- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816406feb0 msgr2=0x7f8164155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816406feb0 0x7f8164155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 -- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 msgr2=0x7f8164154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 0x7f8164154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8163e00640 1 -- 172.21.15.88:0/4152958511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8164156660 con 0x7f816415be30 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.541+0000 7f8169600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 0x7f8164154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:59:26.543 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f8163e00640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f815002c960 tx=0x7f8150004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:26.544 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150042070 con 0x7f816415be30 2024-04-26T19:59:26.544 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150031cb0 con 0x7f816415be30 2024-04-26T19:59:26.544 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8164156940 con 0x7f816415be30 2024-04-26T19:59:26.544 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150036040 con 0x7f816415be30 2024-04-26T19:59:26.545 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.542+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8164156f00 con 0x7f816415be30 2024-04-26T19:59:26.545 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.543+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8150045070 con 0x7f816415be30 2024-04-26T19:59:26.545 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.543+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f8150047b20 con 0x7f816415be30 2024-04-26T19:59:26.545 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.543+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8164069e70 con 0x7f816415be30 2024-04-26T19:59:26.548 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.547+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f815002f070 con 0x7f816415be30 2024-04-26T19:59:26.614 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.614+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f816406e210 con 0x7f816415be30 2024-04-26T19:59:26.615 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.615+0000 7f8160c00640 1 -- 172.21.15.88:0/4152958511 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f815003bde0 con 0x7f816415be30 2024-04-26T19:59:26.616 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-26T19:59:26.616 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":10,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:25.173989+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 msgr2=0x7f816414d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f815002c960 tx=0x7f8150004730 comp rx=0 tx=0).stop 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 shutdown_connections 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f816415c210 0x7f8164154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f816415be30 0x7f816414d7d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 --2- 172.21.15.88:0/4152958511 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f816406feb0 0x7f8164155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 >> 172.21.15.88:0/4152958511 conn(0x7f81640d0c40 msgr2=0x7f8164158070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 shutdown_connections 2024-04-26T19:59:26.617 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:26.616+0000 7f816b600640 1 -- 172.21.15.88:0/4152958511 wait complete. 2024-04-26T19:59:26.629 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T19:59:26.783 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.782+0000 7f0172693040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:59:26.893 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:26.891+0000 7fe1dcd2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:59:26.914 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:26.914+0000 7f0172693040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:59:27.030 INFO:tasks.ceph.mgr.x.smithi132.stderr:2024-04-26T19:59:27.028+0000 7fe1dcd2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:59:27.958 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:27.957+0000 7f0112c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T19:59:27.958 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T19:59:27.958+0000 7f0112c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T19:59:29.384 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-26T19:59:29.382+0000 7fddec60c640 -1 osd.4 0 log_to_monitors true 2024-04-26T19:59:29.509 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-26T19:59:29.509+0000 7f8fefa1c640 -1 osd.0 0 log_to_monitors true 2024-04-26T19:59:29.543 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-26T19:59:29.541+0000 7f91ba916640 -1 osd.6 0 log_to_monitors true 2024-04-26T19:59:29.551 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-26T19:59:29.550+0000 7f18a0220640 -1 osd.7 0 log_to_monitors true 2024-04-26T19:59:29.563 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-26T19:59:29.563+0000 7f8b27f2c640 -1 osd.3 0 log_to_monitors true 2024-04-26T19:59:29.579 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-26T19:59:29.579+0000 7ff3b7921640 -1 osd.2 0 log_to_monitors true 2024-04-26T19:59:29.631 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-26T19:59:29.629+0000 7f3b94c17640 -1 osd.5 0 log_to_monitors true 2024-04-26T19:59:29.747 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-26T19:59:29.746+0000 7ff26f9dc640 -1 osd.1 0 log_to_monitors true 2024-04-26T19:59:33.117 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-26T19:59:33.115+0000 7f189b400640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T19:59:33.131 INFO:tasks.ceph.osd.7.smithi132.stderr:2024-04-26T19:59:33.129+0000 7f1890a00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.224 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-26T19:59:33.223+0000 7f91b6200640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T19:59:33.233 INFO:tasks.ceph.osd.6.smithi132.stderr:2024-04-26T19:59:33.232+0000 7f91ab800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.253 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-26T19:59:33.251+0000 7fdde6e00640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T19:59:33.261 INFO:tasks.ceph.osd.4.smithi132.stderr:2024-04-26T19:59:33.260+0000 7fdddce00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.298 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-26T19:59:33.297+0000 7f3b8fe00640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T19:59:33.303 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-26T19:59:33.303+0000 7ff26a800640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T19:59:33.309 INFO:tasks.ceph.osd.5.smithi132.stderr:2024-04-26T19:59:33.307+0000 7f3b85400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.311 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-04-26T19:59:33.311+0000 7ff260800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.312 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-26T19:59:33.312+0000 7f8fea400640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T19:59:33.325 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-04-26T19:59:33.324+0000 7f8fe0400640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.326 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-26T19:59:33.326+0000 7f8b23800640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T19:59:33.337 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-04-26T19:59:33.337+0000 7f8b18e00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.341 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-26T19:59:33.341+0000 7ff3b3200640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T19:59:33.345 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-04-26T19:59:33.345+0000 7ff3a8800640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:59:33.434 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:33.597 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.596+0000 7fe68d600640 1 Processor -- start 2024-04-26T19:59:33.597 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.596+0000 7fe68d600640 1 -- start start 2024-04-26T19:59:33.598 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe68d600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe68815f260 0x7fe68806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe68d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6880b5910 0x7fe68806dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe68d600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe688072ce0 con 0x7fe68806e510 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe68d600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe68806e980 con 0x7fe68815f260 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe68d600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe688073580 con 0x7fe6880b5910 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe687400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe68806e510 0x7fe6880b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45388/0 (socket says 172.21.15.88:45388) 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe687400640 1 -- 172.21.15.88:0/1193061376 learned_addr learned my addr 172.21.15.88:0/1193061376 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:33.599 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe686a00640 1 --2- 172.21.15.88:0/1193061376 >> v2:172.21.15.88:3300/0 conn(0x7fe68815f260 0x7fe68806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.598+0000 7fe686000640 1 --2- 172.21.15.88:0/1193061376 >> v2:172.21.15.132:3300/0 conn(0x7fe6880b5910 0x7fe68806dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe685600640 1 -- 172.21.15.88:0/1193061376 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718443839 0 0) 0x7fe688072ce0 con 0x7fe68806e510 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe685600640 1 -- 172.21.15.88:0/1193061376 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe66c003520 con 0x7fe68806e510 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe686a00640 1 -- 172.21.15.88:0/1193061376 >> v2:172.21.15.132:3300/0 conn(0x7fe6880b5910 msgr2=0x7fe68806dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe686a00640 1 --2- 172.21.15.88:0/1193061376 >> v2:172.21.15.132:3300/0 conn(0x7fe6880b5910 0x7fe68806dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe686a00640 1 -- 172.21.15.88:0/1193061376 >> v1:172.21.15.88:6789/0 conn(0x7fe68806e510 legacy=0x7fe6880b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.600 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.599+0000 7fe686a00640 1 -- 172.21.15.88:0/1193061376 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe688072fc0 con 0x7fe68815f260 2024-04-26T19:59:33.601 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.600+0000 7fe686a00640 1 --2- 172.21.15.88:0/1193061376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68806fec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe68809f0c0 tx=0x7fe67802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356dee2232a33794 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:33.601 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.600+0000 7fe685600640 1 -- 172.21.15.88:0/1193061376 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678037070 con 0x7fe68815f260 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.600+0000 7fe685600640 1 -- 172.21.15.88:0/1193061376 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67802add0 con 0x7fe68815f260 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.600+0000 7fe685600640 1 -- 172.21.15.88:0/1193061376 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678032660 con 0x7fe68815f260 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 -- 172.21.15.88:0/1193061376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 msgr2=0x7fe68806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 --2- 172.21.15.88:0/1193061376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68806fec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe68809f0c0 tx=0x7fe67802a080 comp rx=0 tx=0).stop 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 -- 172.21.15.88:0/1193061376 shutdown_connections 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 --2- 172.21.15.88:0/1193061376 >> v2:172.21.15.132:3300/0 conn(0x7fe6880b5910 0x7fe68806dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 --2- 172.21.15.88:0/1193061376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68806fec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 -- 172.21.15.88:0/1193061376 >> 172.21.15.88:0/1193061376 conn(0x7fe6880d0c40 msgr2=0x7fe6880d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 -- 172.21.15.88:0/1193061376 shutdown_connections 2024-04-26T19:59:33.602 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.601+0000 7fe68d600640 1 -- 172.21.15.88:0/1193061376 wait complete. 2024-04-26T19:59:33.603 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.602+0000 7fe68d600640 1 Processor -- start 2024-04-26T19:59:33.603 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.603+0000 7fe68d600640 1 -- start start 2024-04-26T19:59:33.605 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68806e510 0x7fe6880af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.605 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6880b5910 0x7fe68814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.605 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880b5d70 con 0x7fe68815f260 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880b3d70 con 0x7fe68806e510 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe68d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe6880b4910 con 0x7fe6880b5910 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42506/0 (socket says 172.21.15.88:42506) 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 -- 172.21.15.88:0/3935167522 learned_addr learned my addr 172.21.15.88:0/3935167522 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe686a00640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68806e510 0x7fe6880af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 -- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6880b5910 msgr2=0x7fe68814a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe686000640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6880b5910 0x7fe68814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6880b5910 0x7fe68814a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 -- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68806e510 msgr2=0x7fe6880af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68806e510 0x7fe6880af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.606 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.604+0000 7fe687400640 1 -- 172.21.15.88:0/3935167522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68809d430 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe687400640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe68000cb40 tx=0x7fe68000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680017070 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68000aca0 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6880cadf0 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680012620 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.605+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68809d990 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.606+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe688069e70 con 0x7fe68815f260 2024-04-26T19:59:33.607 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.607+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe68001a070 con 0x7fe68815f260 2024-04-26T19:59:33.608 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.608+0000 7fe67ea00640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 0x7fe66806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:33.608 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.608+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fe68008abe0 con 0x7fe68815f260 2024-04-26T19:59:33.608 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.608+0000 7fe686a00640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 0x7fe66806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:33.609 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.608+0000 7fe686a00640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 0x7fe66806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe678034000 tx=0x7fe67802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:33.613 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.613+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe68809d430 con 0x7fe68815f260 2024-04-26T19:59:33.800 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.799+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6880d1780 con 0x7fe68815f260 2024-04-26T19:59:33.801 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.800+0000 7fe67ea00640 1 -- 172.21.15.88:0/3935167522 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fe680059de0 con 0x7fe68815f260 2024-04-26T19:59:33.801 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-26T19:59:33.801 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":12,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:30.961544+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:33.804 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.803+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 msgr2=0x7fe66806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.804 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.803+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 0x7fe66806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe678034000 tx=0x7fe67802e000 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 msgr2=0x7fe68809e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe68000cb40 tx=0x7fe68000a060 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 shutdown_connections 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe68815f260 0x7fe68809e190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe66806c860 0x7fe66806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6880b5910 0x7fe68814a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 --2- 172.21.15.88:0/3935167522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe68806e510 0x7fe6880af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 >> 172.21.15.88:0/3935167522 conn(0x7fe6880d0c40 msgr2=0x7fe68815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 shutdown_connections 2024-04-26T19:59:33.805 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:33.804+0000 7fe68d600640 1 -- 172.21.15.88:0/3935167522 wait complete. 2024-04-26T19:59:33.817 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T19:59:39.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:59:39.254 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:59:40.624 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:40.806 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.804+0000 7f49b5c00640 1 Processor -- start 2024-04-26T19:59:40.806 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.805+0000 7f49b5c00640 1 -- start start 2024-04-26T19:59:40.808 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49b5c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49b015aaa0 0x7f49b00add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.808 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49b5c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49b00b0040 0x7f49b006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.808 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49b5c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b0161a00 con 0x7f49b015dfc0 2024-04-26T19:59:40.808 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49b5c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f49b00ae430 con 0x7f49b00b0040 2024-04-26T19:59:40.808 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49b5c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f49b0160c40 con 0x7f49b015aaa0 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49af400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f49b00b0040 0x7f49b006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49a7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f49b015dfc0 0x7f49b00af930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45202/0 (socket says 172.21.15.88:45202) 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.806+0000 7f49a7e00640 1 -- 172.21.15.88:0/1644010167 learned_addr learned my addr 172.21.15.88:0/1644010167 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49aea00640 1 --2- 172.21.15.88:0/1644010167 >> v2:172.21.15.132:3300/0 conn(0x7f49b015aaa0 0x7f49b00add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49af400640 1 -- 172.21.15.88:0/1644010167 >> v2:172.21.15.132:3300/0 conn(0x7f49b015aaa0 msgr2=0x7f49b00add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49af400640 1 --2- 172.21.15.88:0/1644010167 >> v2:172.21.15.132:3300/0 conn(0x7f49b015aaa0 0x7f49b00add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49af400640 1 -- 172.21.15.88:0/1644010167 >> v1:172.21.15.88:6789/0 conn(0x7f49b015dfc0 legacy=0x7f49b00af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49af400640 1 -- 172.21.15.88:0/1644010167 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0160f20 con 0x7f49b00b0040 2024-04-26T19:59:40.809 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.807+0000 7f49ae000640 1 -- 172.21.15.88:0/1644010167 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539758716 0 0) 0x7f49b0161a00 con 0x7f49b015dfc0 2024-04-26T19:59:40.810 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.808+0000 7f49af400640 1 --2- 172.21.15.88:0/1644010167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b00b0040 0x7f49b006f280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49b0074800 tx=0x7f49a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f13df0d46e41c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:40.810 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.808+0000 7f49ae000640 1 -- 172.21.15.88:0/1644010167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8007740 con 0x7f49b00b0040 2024-04-26T19:59:40.810 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.808+0000 7f49ae000640 1 -- 172.21.15.88:0/1644010167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a80078e0 con 0x7f49b00b0040 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.808+0000 7f49ae000640 1 -- 172.21.15.88:0/1644010167 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8007cb0 con 0x7f49b00b0040 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 -- 172.21.15.88:0/1644010167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b00b0040 msgr2=0x7f49b006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 --2- 172.21.15.88:0/1644010167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b00b0040 0x7f49b006f280 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49b0074800 tx=0x7f49a802a5f0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 -- 172.21.15.88:0/1644010167 shutdown_connections 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 --2- 172.21.15.88:0/1644010167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b00b0040 0x7f49b006f280 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 --2- 172.21.15.88:0/1644010167 >> v2:172.21.15.132:3300/0 conn(0x7f49b015aaa0 0x7f49b00add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 -- 172.21.15.88:0/1644010167 >> 172.21.15.88:0/1644010167 conn(0x7f49b00d0c40 msgr2=0x7f49b00d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 -- 172.21.15.88:0/1644010167 shutdown_connections 2024-04-26T19:59:40.811 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.809+0000 7f49b5c00640 1 -- 172.21.15.88:0/1644010167 wait complete. 2024-04-26T19:59:40.812 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.810+0000 7f49b5c00640 1 Processor -- start 2024-04-26T19:59:40.812 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.811+0000 7f49b5c00640 1 -- start start 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b00b0040 0x7f49b01f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b015dfc0 0x7f49b0153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015e450 con 0x7f49b015aaa0 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f49b01614e0 con 0x7f49b015dfc0 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49b5c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f49b015ae80 con 0x7f49b00b0040 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58074/0 (socket says 172.21.15.88:58074) 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 -- 172.21.15.88:0/2902983987 learned_addr learned my addr 172.21.15.88:0/2902983987 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49aea00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b00b0040 0x7f49b01f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49af400640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b015dfc0 0x7f49b0153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 -- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b00b0040 msgr2=0x7f49b01f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.815 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b00b0040 0x7f49b01f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 -- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b015dfc0 msgr2=0x7f49b0153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b015dfc0 0x7f49b0153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.812+0000 7f49a7e00640 1 -- 172.21.15.88:0/2902983987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b015ec50 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49a7e00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f499800c460 tx=0x7f499800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998018070 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4998007500 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998004a20 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b015e970 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.813+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b0153660 con 0x7f49b015aaa0 2024-04-26T19:59:40.816 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.814+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b0069e70 con 0x7f49b015aaa0 2024-04-26T19:59:40.817 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.815+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4998004bc0 con 0x7f49b015aaa0 2024-04-26T19:59:40.818 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.816+0000 7f49a6a00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 0x7f498c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:40.818 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.816+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f499808ab10 con 0x7f49b015aaa0 2024-04-26T19:59:40.818 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.817+0000 7f49aea00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 0x7f498c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:40.819 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.817+0000 7f49aea00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 0x7f498c06edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49a0006fd0 tx=0x7f49a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:40.823 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:40.822+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4998059290 con 0x7f49b015aaa0 2024-04-26T19:59:41.026 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.024+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49b00d19d0 con 0x7f49b015aaa0 2024-04-26T19:59:41.028 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.026+0000 7f49a6a00640 1 -- 172.21.15.88:0/2902983987 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f4998058b90 con 0x7f49b015aaa0 2024-04-26T19:59:41.028 INFO:teuthology.misc.health.smithi088.stdout: 2024-04-26T19:59:41.028 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":16,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:37.172243+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:41.031 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 msgr2=0x7f498c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.031 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 0x7f498c06edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49a0006fd0 tx=0x7f49a0008040 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 msgr2=0x7f49b00b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f499800c460 tx=0x7f499800c920 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 shutdown_connections 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b015dfc0 0x7f49b0153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b015aaa0 0x7f49b00b1bf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f498c06c8f0 0x7f498c06edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.029+0000 7f49b5c00640 1 --2- 172.21.15.88:0/2902983987 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49b00b0040 0x7f49b01f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.030+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 >> 172.21.15.88:0/2902983987 conn(0x7f49b00d0c40 msgr2=0x7f49b0155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.030+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 shutdown_connections 2024-04-26T19:59:41.032 INFO:teuthology.misc.health.smithi088.stderr:2024-04-26T19:59:41.030+0000 7f49b5c00640 1 -- 172.21.15.88:0/2902983987 wait complete. 2024-04-26T19:59:41.047 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T19:59:41.047 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T19:59:41.047 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T19:59:41.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T19:59:41.052 INFO:tasks.ceph.mds.a:Started 2024-04-26T19:59:41.052 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T19:59:41.053 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T19:59:41.057 INFO:tasks.ceph.mds.c:Started 2024-04-26T19:59:41.057 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T19:59:41.058 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T19:59:41.065 INFO:tasks.ceph.mds.b:Started 2024-04-26T19:59:41.065 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T19:59:41.066 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T19:59:41.072 INFO:tasks.ceph.mds.d:Started 2024-04-26T19:59:41.073 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T19:59:41.073 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:41.199 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-04-26T19:59:41.200 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T19:59:41.218 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-26T19:59:41.221 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-26T19:59:41.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.246+0000 7f5bf7200640 1 Processor -- start 2024-04-26T19:59:41.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.246+0000 7f5bf7200640 1 -- start start 2024-04-26T19:59:41.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.247+0000 7f5bf7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5bf815f1b0 0x7f5bf80d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.247+0000 7f5bf7200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bf8163650 0x7f5bf80b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf7200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf80d3f50 con 0x7f5bf8162b00 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf7200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80d36b0 con 0x7f5bf8163650 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80d3990 con 0x7f5bf815f1b0 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bf8163650 0x7f5bf80b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5bf8163650 0x7f5bf80b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58106/0 (socket says 172.21.15.88:58106) 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 -- 172.21.15.88:0/3683786867 learned_addr learned my addr 172.21.15.88:0/3683786867 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:41.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf5e00640 1 --2- 172.21.15.88:0/3683786867 >> v2:172.21.15.132:3300/0 conn(0x7f5bf815f1b0 0x7f5bf80d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 -- 172.21.15.88:0/3683786867 >> v2:172.21.15.132:3300/0 conn(0x7f5bf815f1b0 msgr2=0x7f5bf80d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:41.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 --2- 172.21.15.88:0/3683786867 >> v2:172.21.15.132:3300/0 conn(0x7f5bf815f1b0 0x7f5bf80d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 -- 172.21.15.88:0/3683786867 >> v1:172.21.15.132:6789/0 conn(0x7f5bf8162b00 legacy=0x7f5bf8162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.248+0000 7f5bf6800640 1 -- 172.21.15.88:0/3683786867 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80d3c70 con 0x7f5bf8163650 2024-04-26T19:59:41.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.249+0000 7f5bf6800640 1 --2- 172.21.15.88:0/3683786867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8163650 0x7f5bf80b82b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5bf806f570 tx=0x7f5bf002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a88223034d9c4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf5400640 1 -- 172.21.15.88:0/3683786867 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0036070 con 0x7f5bf8163650 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf5400640 1 -- 172.21.15.88:0/3683786867 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0031600 con 0x7f5bf8163650 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf5400640 1 -- 172.21.15.88:0/3683786867 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00319d0 con 0x7f5bf8163650 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 -- 172.21.15.88:0/3683786867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8163650 msgr2=0x7f5bf80b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 --2- 172.21.15.88:0/3683786867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8163650 0x7f5bf80b82b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5bf806f570 tx=0x7f5bf002a730 comp rx=0 tx=0).stop 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 -- 172.21.15.88:0/3683786867 shutdown_connections 2024-04-26T19:59:41.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 --2- 172.21.15.88:0/3683786867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8163650 0x7f5bf80b82b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 --2- 172.21.15.88:0/3683786867 >> v2:172.21.15.132:3300/0 conn(0x7f5bf815f1b0 0x7f5bf80d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 -- 172.21.15.88:0/3683786867 >> 172.21.15.88:0/3683786867 conn(0x7f5bf809e230 msgr2=0x7f5bf80d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 -- 172.21.15.88:0/3683786867 shutdown_connections 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.250+0000 7f5bf7200640 1 -- 172.21.15.88:0/3683786867 wait complete. 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.252+0000 7f5bf7200640 1 Processor -- start 2024-04-26T19:59:41.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.252+0000 7f5bf7200640 1 -- start start 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf815f1b0 0x7f5bf815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 0x7f5bf806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80b87f0 con 0x7f5bf8162b00 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d33a0 con 0x7f5bf8163650 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf80b8970 con 0x7f5bf815f1b0 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf6800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:50818/0 (socket says 172.21.15.88:50818) 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 0x7f5bf806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf6800640 1 -- 172.21.15.88:0/1334157268 learned_addr learned my addr 172.21.15.88:0/1334157268 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:41.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.253+0000 7f5bf5e00640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf815f1b0 0x7f5bf815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 -- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf815f1b0 msgr2=0x7f5bf815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf815f1b0 0x7f5bf815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 -- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 msgr2=0x7f5bf806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 -- 172.21.15.88:0/1334157268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf81623f0 con 0x7f5bf8162b00 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bf6800640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bed400640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 0x7f5bf806d1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5be000c370 tx=0x7f5be000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00052d0 con 0x7f5bf8162b00 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be0005490 con 0x7f5bf8162b00 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0007260 con 0x7f5bf8162b00 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.254+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80c89e0 con 0x7f5bf8162b00 2024-04-26T19:59:41.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.255+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf80c8fa0 con 0x7f5bf8162b00 2024-04-26T19:59:41.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.256+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf80693c0 con 0x7f5bf8162b00 2024-04-26T19:59:41.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.257+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5be0007400 con 0x7f5bf8162b00 2024-04-26T19:59:41.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.258+0000 7f5beea00640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 0x7f5bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.258+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5be008abc0 con 0x7f5bf8162b00 2024-04-26T19:59:41.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.258+0000 7f5bf5e00640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 0x7f5bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.259+0000 7f5bf5e00640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 0x7f5bd406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009b40 tx=0x7f5be8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.263+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5be0058d80 con 0x7f5bf8162b00 2024-04-26T19:59:41.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.452+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bf808d090 con 0x7f5bf8162b00 2024-04-26T19:59:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.454+0000 7f5beea00640 1 -- 172.21.15.88:0/1334157268 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f5be005d080 con 0x7f5bf8162b00 2024-04-26T19:59:41.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:41.456 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":16,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:37.172243+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 msgr2=0x7f5bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 0x7f5bd406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5be8009b40 tx=0x7f5be8009190 comp rx=0 tx=0).stop 2024-04-26T19:59:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 msgr2=0x7f5bf806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 0x7f5bf806d1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5be000c370 tx=0x7f5be000c830 comp rx=0 tx=0).stop 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 shutdown_connections 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5bf8163650 0x7f5bf806d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bf8162b00 0x7f5bf806d1b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5bd406c8f0 0x7f5bd406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 --2- 172.21.15.88:0/1334157268 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5bf815f1b0 0x7f5bf815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.456+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 >> 172.21.15.88:0/1334157268 conn(0x7f5bf809e230 msgr2=0x7f5bf80b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.457+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 shutdown_connections 2024-04-26T19:59:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.457+0000 7f5bf7200640 1 -- 172.21.15.88:0/1334157268 wait complete. 2024-04-26T19:59:41.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:59:41.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:59:41.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.668+0000 7f093ce00640 1 Processor -- start 2024-04-26T19:59:41.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.668+0000 7f093ce00640 1 -- start start 2024-04-26T19:59:41.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f093ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f09380b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f093ce00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0938072970 0x7f09380ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f093ce00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f093806e320 con 0x7f09380b4ed0 2024-04-26T19:59:41.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f093ce00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f093806e600 con 0x7f0938072970 2024-04-26T19:59:41.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f093ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f093806e8e0 con 0x7f0938071e80 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f0937e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0938072970 0x7f09380ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f0937e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0938072970 0x7f09380ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58134/0 (socket says 172.21.15.88:58134) 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f0937e00640 1 -- 172.21.15.88:0/332780568 learned_addr learned my addr 172.21.15.88:0/332780568 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f0936a00640 1 --1- 172.21.15.88:0/332780568 >> v1:172.21.15.88:6789/0 conn(0x7f09380b4ed0 0x7f0938072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45206/0 (socket says 172.21.15.88:45206) 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.670+0000 7f0937400640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f09380b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.671+0000 7f0937e00640 1 -- 172.21.15.88:0/332780568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 msgr2=0x7f09380b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.671+0000 7f0937e00640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f09380b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.671+0000 7f0937e00640 1 -- 172.21.15.88:0/332780568 >> v1:172.21.15.88:6789/0 conn(0x7f09380b4ed0 legacy=0x7f0938072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.671+0000 7f0937e00640 1 -- 172.21.15.88:0/332780568 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093806ebc0 con 0x7f0938072970 2024-04-26T19:59:41.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.671+0000 7f0936000640 1 -- 172.21.15.88:0/332780568 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004373924 0 0) 0x7f093806e320 con 0x7f09380b4ed0 2024-04-26T19:59:41.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f0937e00640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f09380ae8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930009e50 tx=0x7f093002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b879d3deaaa525e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f0936000640 1 -- 172.21.15.88:0/332780568 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930036070 con 0x7f0938072970 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f0936000640 1 -- 172.21.15.88:0/332780568 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0930031600 con 0x7f0938072970 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f0936000640 1 -- 172.21.15.88:0/332780568 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300319d0 con 0x7f0938072970 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f093ce00640 1 -- 172.21.15.88:0/332780568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 msgr2=0x7f09380ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.672+0000 7f093ce00640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f09380ae8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930009e50 tx=0x7f093002a710 comp rx=0 tx=0).stop 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 -- 172.21.15.88:0/332780568 shutdown_connections 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f09380ae8c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 --2- 172.21.15.88:0/332780568 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f09380b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 -- 172.21.15.88:0/332780568 >> 172.21.15.88:0/332780568 conn(0x7f09380b2900 msgr2=0x7f09380b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 -- 172.21.15.88:0/332780568 shutdown_connections 2024-04-26T19:59:41.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.673+0000 7f093ce00640 1 -- 172.21.15.88:0/332780568 wait complete. 2024-04-26T19:59:41.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.674+0000 7f093ce00640 1 Processor -- start 2024-04-26T19:59:41.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.675+0000 7f093ce00640 1 -- start start 2024-04-26T19:59:41.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.675+0000 7f093ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f0938161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.675+0000 7f093ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f093ce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09380b4ed0 0x7f0938158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f093ce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f093806f0e0 con 0x7f0938072970 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f093ce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f09380b5ff0 con 0x7f09380b4ed0 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f093ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f093806f260 con 0x7f0938071e80 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58142/0 (socket says 172.21.15.88:58142) 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 -- 172.21.15.88:0/3228609089 learned_addr learned my addr 172.21.15.88:0/3228609089 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0937400640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f0938161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 -- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 msgr2=0x7f0938161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f0938161af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 -- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09380b4ed0 msgr2=0x7f0938158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09380b4ed0 0x7f0938158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 -- 172.21.15.88:0/3228609089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09381569d0 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.676+0000 7f0936a00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f092400c430 tx=0x7f092400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.677+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240074e0 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.677+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09240047e0 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.677+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924004bd0 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.677+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0938156cb0 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.677+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0938157270 con 0x7f0938072970 2024-04-26T19:59:41.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.678+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09380689f0 con 0x7f0938072970 2024-04-26T19:59:41.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.679+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0924011070 con 0x7f0938072970 2024-04-26T19:59:41.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.680+0000 7f092f400640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 0x7f091806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:41.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.681+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f092408b580 con 0x7f0938072970 2024-04-26T19:59:41.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.681+0000 7f0937400640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 0x7f091806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:41.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.681+0000 7f0937400640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 0x7f091806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0928006fd0 tx=0x7f0928008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:41.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.685+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09380689f0 con 0x7f0938072970 2024-04-26T19:59:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.885+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09380b4740 con 0x7f0938072970 2024-04-26T19:59:41.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.886+0000 7f092f400640 1 -- 172.21.15.88:0/3228609089 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f09380b4740 con 0x7f0938072970 2024-04-26T19:59:41.888 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.888+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 msgr2=0x7f091806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.888+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 0x7f091806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0928006fd0 tx=0x7f0928008040 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 msgr2=0x7f0938157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f092400c430 tx=0x7f092400c8f0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 shutdown_connections 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09380b4ed0 0x7f0938158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0938072970 0x7f0938157b00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f091806c8f0 0x7f091806edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 --2- 172.21.15.88:0/3228609089 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0938071e80 0x7f0938161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 >> 172.21.15.88:0/3228609089 conn(0x7f09380b2900 msgr2=0x7f09381557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 shutdown_connections 2024-04-26T19:59:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:41.889+0000 7f093ce00640 1 -- 172.21.15.88:0/3228609089 wait complete. 2024-04-26T19:59:41.909 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:59:42.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.095+0000 7f8193a00640 1 Processor -- start 2024-04-26T19:59:42.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.095+0000 7f8193a00640 1 -- start start 2024-04-26T19:59:42.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8193a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f818c1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8193a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f818c15f740 0x7f818c15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8193a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f818c0b5fb0 con 0x7f818c1598a0 2024-04-26T19:59:42.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8193a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f818c159c80 con 0x7f818c15f740 2024-04-26T19:59:42.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8193a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f818c160110 con 0x7f818c06ce80 2024-04-26T19:59:42.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8191000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f818c1598a0 0x7f818c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45222/0 (socket says 172.21.15.88:45222) 2024-04-26T19:59:42.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.097+0000 7f8191000640 1 -- 172.21.15.88:0/1516425981 learned_addr learned my addr 172.21.15.88:0/1516425981 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516027430 0 0) 0x7f818c0b5fb0 con 0x7f818c1598a0 2024-04-26T19:59:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8170003520 con 0x7f818c1598a0 2024-04-26T19:59:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f8191a00640 1 --2- 172.21.15.88:0/1516425981 >> v2:172.21.15.132:3300/0 conn(0x7f818c15f740 0x7f818c15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f8183e00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f818c1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3949643297 0 0) 0x7f8170003520 con 0x7f818c1598a0 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 msgr2=0x7f818c1638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f818c1638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 >> v2:172.21.15.132:3300/0 conn(0x7f818c15f740 msgr2=0x7f818c15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 --2- 172.21.15.88:0/1516425981 >> v2:172.21.15.132:3300/0 conn(0x7f818c15f740 0x7f818c15fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c160370 con 0x7f818c1598a0 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f817c001670 con 0x7f818c1598a0 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 >> v1:172.21.15.88:6789/0 conn(0x7f818c1598a0 legacy=0x7f818c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81700039e0 0x7f8170003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f8170004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8170004d40 0x7f81700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f818c159c80 con 0x7f81700039e0 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f818c160110 con 0x7f8170004d40 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f817000bb40 con 0x7f818c06ce80 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.098+0000 7f8191000640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81700039e0 0x7f8170003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8183e00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f8170004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191000640 1 -- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 msgr2=0x7f8170004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191a00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8170004d40 0x7f81700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191000640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c06ce80 0x7f8170004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191000640 1 -- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8170004d40 msgr2=0x7f81700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191000640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8170004d40 0x7f81700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.099+0000 7f8191000640 1 -- 172.21.15.88:0/1516425981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8170003520 con 0x7f81700039e0 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.100+0000 7f8191000640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81700039e0 0x7f8170003e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c240 tx=0x7f8170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a84e69e0c89c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.100+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c007d10 con 0x7f81700039e0 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.100+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f817c005ad0 con 0x7f81700039e0 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.100+0000 7f818be00640 1 -- 172.21.15.88:0/1516425981 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c007070 con 0x7f81700039e0 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 -- 172.21.15.88:0/1516425981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81700039e0 msgr2=0x7f8170003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 --2- 172.21.15.88:0/1516425981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81700039e0 0x7f8170003e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c240 tx=0x7f8170005830 comp rx=0 tx=0).stop 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8191a00640 1 -- 172.21.15.88:0/1516425981 reap_dead start 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 -- 172.21.15.88:0/1516425981 shutdown_connections 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 -- 172.21.15.88:0/1516425981 >> 172.21.15.88:0/1516425981 conn(0x7f818c0b2d50 msgr2=0x7f818c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 -- 172.21.15.88:0/1516425981 shutdown_connections 2024-04-26T19:59:42.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.101+0000 7f8193a00640 1 -- 172.21.15.88:0/1516425981 wait complete. 2024-04-26T19:59:42.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.103+0000 7f8193a00640 1 Processor -- start 2024-04-26T19:59:42.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.103+0000 7f8193a00640 1 -- start start 2024-04-26T19:59:42.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81700039e0 0x7f818c0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c0cbec0 0x7f818c0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f818c163390 con 0x7f818c06e7b0 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1630b0 con 0x7f81700039e0 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8193a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f818c160650 con 0x7f818c0cbec0 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58166/0 (socket says 172.21.15.88:58166) 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 -- 172.21.15.88:0/665153508 learned_addr learned my addr 172.21.15.88:0/665153508 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191000640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81700039e0 0x7f818c0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 -- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c0cbec0 msgr2=0x7f818c0cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c0cbec0 0x7f818c0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 -- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81700039e0 msgr2=0x7f818c0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81700039e0 0x7f818c0ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 -- 172.21.15.88:0/665153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c0cc880 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.104+0000 7f8191a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f818c0b3980 tx=0x7f8184009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.105+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184019070 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.105+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8184007ce0 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.105+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184014640 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.105+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c148890 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.105+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c0cce10 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.106+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c069540 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.107+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f818401c070 con 0x7f818c06e7b0 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.108+0000 7f8189600640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 0x7f816806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.108+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f818408b390 con 0x7f818c06e7b0 2024-04-26T19:59:42.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.108+0000 7f8191000640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 0x7f816806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.109+0000 7f8191000640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 0x7f816806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f817c033000 tx=0x7f817c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.112+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8184059a70 con 0x7f818c06e7b0 2024-04-26T19:59:42.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.356+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818c161e50 con 0x7f818c06e7b0 2024-04-26T19:59:42.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.357+0000 7f8189600640 1 -- 172.21.15.88:0/665153508 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f8184059370 con 0x7f818c06e7b0 2024-04-26T19:59:42.359 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:42.359 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T19:59:42.360 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 3 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 msgr2=0x7f816806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 0x7f816806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f817c033000 tx=0x7f817c02e000 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 msgr2=0x7f818c06ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f818c0b3980 tx=0x7f8184009d40 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 shutdown_connections 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f818c06e7b0 0x7f818c06ec30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f816806c8f0 0x7f816806edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f818c0cbec0 0x7f818c0cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 --2- 172.21.15.88:0/665153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81700039e0 0x7f818c0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 >> 172.21.15.88:0/665153508 conn(0x7f818c0b2d50 msgr2=0x7f818c06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:42.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 shutdown_connections 2024-04-26T19:59:42.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.360+0000 7f8193a00640 1 -- 172.21.15.88:0/665153508 wait complete. 2024-04-26T19:59:42.380 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.571+0000 7f95aec00640 1 Processor -- start 2024-04-26T19:59:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.571+0000 7f95aec00640 1 -- start start 2024-04-26T19:59:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.572+0000 7f95aec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.572+0000 7f95aec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f95a80b3b20 0x7f95a80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.572+0000 7f95aec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a8146a60 con 0x7f95a8146680 2024-04-26T19:59:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95aec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f95a806e970 con 0x7f95a80b3b20 2024-04-26T19:59:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95aec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f95a80b60a0 con 0x7f95a80d0e70 2024-04-26T19:59:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95ace00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95ace00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:46064/0 (socket says 172.21.15.88:46064) 2024-04-26T19:59:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95ace00640 1 -- 172.21.15.88:0/810810696 learned_addr learned my addr 172.21.15.88:0/810810696 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:42.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.573+0000 7f95ae200640 1 --2- 172.21.15.88:0/810810696 >> v2:172.21.15.132:3300/0 conn(0x7f95a80b3b20 0x7f95a80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.574+0000 7f95ace00640 1 -- 172.21.15.88:0/810810696 >> v2:172.21.15.132:3300/0 conn(0x7f95a80b3b20 msgr2=0x7f95a80b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.574+0000 7f95ace00640 1 --2- 172.21.15.88:0/810810696 >> v2:172.21.15.132:3300/0 conn(0x7f95a80b3b20 0x7f95a80b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.574+0000 7f95ace00640 1 -- 172.21.15.88:0/810810696 >> v1:172.21.15.132:6789/0 conn(0x7f95a8146680 legacy=0x7f95a80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.574+0000 7f95ace00640 1 -- 172.21.15.88:0/810810696 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a806dd60 con 0x7f95a80d0e70 2024-04-26T19:59:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.575+0000 7f95ace00640 1 --2- 172.21.15.88:0/810810696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9594009870 tx=0x7f959402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5e74c34710299e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.576+0000 7f95a7e00640 1 -- 172.21.15.88:0/810810696 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594037070 con 0x7f95a80d0e70 2024-04-26T19:59:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.576+0000 7f95a7e00640 1 -- 172.21.15.88:0/810810696 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594032440 con 0x7f95a80d0e70 2024-04-26T19:59:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.576+0000 7f95a7e00640 1 -- 172.21.15.88:0/810810696 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594032810 con 0x7f95a80d0e70 2024-04-26T19:59:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 -- 172.21.15.88:0/810810696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 msgr2=0x7f95a80d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 --2- 172.21.15.88:0/810810696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9594009870 tx=0x7f959402a270 comp rx=0 tx=0).stop 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 -- 172.21.15.88:0/810810696 shutdown_connections 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 --2- 172.21.15.88:0/810810696 >> v2:172.21.15.132:3300/0 conn(0x7f95a80b3b20 0x7f95a80b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 --2- 172.21.15.88:0/810810696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80d0e70 0x7f95a80d12b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 -- 172.21.15.88:0/810810696 >> 172.21.15.88:0/810810696 conn(0x7f95a806c740 msgr2=0x7f95a806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 -- 172.21.15.88:0/810810696 shutdown_connections 2024-04-26T19:59:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.577+0000 7f95aec00640 1 -- 172.21.15.88:0/810810696 wait complete. 2024-04-26T19:59:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.579+0000 7f95aec00640 1 Processor -- start 2024-04-26T19:59:42.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.579+0000 7f95aec00640 1 -- start start 2024-04-26T19:59:42.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80b3b20 0x7f95a8161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a8146680 0x7f95a813c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f95a806e320 con 0x7f95a80d0e70 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80d17f0 con 0x7f95a8146680 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.580+0000 7f95aec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f95a806e600 con 0x7f95a80b3b20 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58182/0 (socket says 172.21.15.88:58182) 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 -- 172.21.15.88:0/837113271 learned_addr learned my addr 172.21.15.88:0/837113271 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ae200640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a8146680 0x7f95a813c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ad800640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80b3b20 0x7f95a8161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 -- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80b3b20 msgr2=0x7f95a8161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80b3b20 0x7f95a8161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 -- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a8146680 msgr2=0x7f95a813c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a8146680 0x7f95a813c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.581+0000 7f95ace00640 1 -- 172.21.15.88:0/837113271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a815fa80 con 0x7f95a80d0e70 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95ace00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f95940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594037040 con 0x7f95a80d0e70 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9594042070 con 0x7f95a80d0e70 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a815fd60 con 0x7f95a80d0e70 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594030880 con 0x7f95a80d0e70 2024-04-26T19:59:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.582+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a8160320 con 0x7f95a80d0e70 2024-04-26T19:59:42.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.583+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a80693c0 con 0x7f95a80d0e70 2024-04-26T19:59:42.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.584+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9594030a20 con 0x7f95a80d0e70 2024-04-26T19:59:42.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.585+0000 7f95a5600640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 0x7f958806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.585+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f95940abff0 con 0x7f95a80d0e70 2024-04-26T19:59:42.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.585+0000 7f95ad800640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 0x7f958806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.586+0000 7f95ad800640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 0x7f958806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f959c001ca0 tx=0x7f959c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:42.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.589+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f959407a7c0 con 0x7f95a80d0e70 2024-04-26T19:59:42.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.785+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95a8141d40 con 0x7f95a80d0e70 2024-04-26T19:59:42.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.786+0000 7f95a5600640 1 -- 172.21.15.88:0/837113271 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12872 (secure 0 0 0) 0x7f959407a0c0 con 0x7f95a80d0e70 2024-04-26T19:59:42.788 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:42.789 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":16,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:37.172243+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 msgr2=0x7f958806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 0x7f958806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f959c001ca0 tx=0x7f959c009040 comp rx=0 tx=0).stop 2024-04-26T19:59:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 msgr2=0x7f95a815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9594000c00 tx=0x7f95940046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 shutdown_connections 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95a8146680 0x7f95a813c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95a80d0e70 0x7f95a815c770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f958806c8f0 0x7f958806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 --2- 172.21.15.88:0/837113271 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f95a80b3b20 0x7f95a8161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.789+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 >> 172.21.15.88:0/837113271 conn(0x7f95a806c740 msgr2=0x7f95a80cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.790+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 shutdown_connections 2024-04-26T19:59:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.790+0000 7f95aec00640 1 -- 172.21.15.88:0/837113271 wait complete. 2024-04-26T19:59:42.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:59:42.809 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:59:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.994+0000 7f53a2400640 1 Processor -- start 2024-04-26T19:59:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.994+0000 7f53a2400640 1 -- start start 2024-04-26T19:59:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f53a2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c06ce80 0x7f539c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f53a2400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f539c0b5ca0 0x7f539c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f53a2400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c0af800 con 0x7f539c15d100 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f53a2400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f539c0ae340 con 0x7f539c0b5ca0 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f53a2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0afaa0 con 0x7f539c06ce80 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.996+0000 7f539be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f539c0b5ca0 0x7f539c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f539c0b5ca0 0x7f539c0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58188/0 (socket says 172.21.15.88:58188) 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 -- 172.21.15.88:0/2848524641 learned_addr learned my addr 172.21.15.88:0/2848524641 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f5393e00640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c06ce80 0x7f539c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 -- 172.21.15.88:0/2848524641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c06ce80 msgr2=0x7f539c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c06ce80 0x7f539c0b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 -- 172.21.15.88:0/2848524641 >> v1:172.21.15.132:6789/0 conn(0x7f539c15d100 legacy=0x7f539c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.997+0000 7f539be00640 1 -- 172.21.15.88:0/2848524641 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0afd80 con 0x7f539c0b5ca0 2024-04-26T19:59:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.998+0000 7f539be00640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c0b4170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5394009e50 tx=0x7f539402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8261c9b5ba7dfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f539aa00640 1 -- 172.21.15.88:0/2848524641 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394036070 con 0x7f539c0b5ca0 2024-04-26T19:59:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f539aa00640 1 -- 172.21.15.88:0/2848524641 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394031600 con 0x7f539c0b5ca0 2024-04-26T19:59:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f539aa00640 1 -- 172.21.15.88:0/2848524641 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940319d0 con 0x7f539c0b5ca0 2024-04-26T19:59:43.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 -- 172.21.15.88:0/2848524641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 msgr2=0x7f539c0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c0b4170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5394009e50 tx=0x7f539402a710 comp rx=0 tx=0).stop 2024-04-26T19:59:43.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 -- 172.21.15.88:0/2848524641 shutdown_connections 2024-04-26T19:59:43.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c0b4170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 --2- 172.21.15.88:0/2848524641 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c06ce80 0x7f539c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:42.999+0000 7f53a2400640 1 -- 172.21.15.88:0/2848524641 >> 172.21.15.88:0/2848524641 conn(0x7f539c0b2d50 msgr2=0x7f539c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.000+0000 7f53a2400640 1 -- 172.21.15.88:0/2848524641 shutdown_connections 2024-04-26T19:59:43.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.000+0000 7f53a2400640 1 -- 172.21.15.88:0/2848524641 wait complete. 2024-04-26T19:59:43.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.001+0000 7f53a2400640 1 Processor -- start 2024-04-26T19:59:43.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.001+0000 7f53a2400640 1 -- start start 2024-04-26T19:59:43.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c06ce80 0x7f539c14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c15d100 0x7f539c09cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f539c09ee50 con 0x7f539c0b5ca0 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0b0340 con 0x7f539c06ce80 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f53a2400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f539c160b70 con 0x7f539c15d100 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f5393e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f5393e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58204/0 (socket says 172.21.15.88:58204) 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.002+0000 7f5393e00640 1 -- 172.21.15.88:0/3009503652 learned_addr learned my addr 172.21.15.88:0/3009503652 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 -- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c15d100 msgr2=0x7f539c09cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c15d100 0x7f539c09cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 -- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c06ce80 msgr2=0x7f539c14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c06ce80 0x7f539c14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 -- 172.21.15.88:0/3009503652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c15fb70 con 0x7f539c0b5ca0 2024-04-26T19:59:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393e00640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f538400d800 tx=0x7f538400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400b6c0 con 0x7f539c0b5ca0 2024-04-26T19:59:43.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c15fe50 con 0x7f539c0b5ca0 2024-04-26T19:59:43.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384019070 con 0x7f539c0b5ca0 2024-04-26T19:59:43.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400bac0 con 0x7f539c0b5ca0 2024-04-26T19:59:43.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.003+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c160410 con 0x7f539c0b5ca0 2024-04-26T19:59:43.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.005+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c069650 con 0x7f539c0b5ca0 2024-04-26T19:59:43.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.005+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f538400bc60 con 0x7f539c0b5ca0 2024-04-26T19:59:43.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.006+0000 7f5393400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 0x7f537806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.006+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f538408bda0 con 0x7f539c0b5ca0 2024-04-26T19:59:43.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.006+0000 7f539b400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 0x7f537806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.007+0000 7f539b400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 0x7f537806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f538c000f60 tx=0x7f538c0079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.011+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f538405a480 con 0x7f539c0b5ca0 2024-04-26T19:59:43.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.200+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f539c073d00 con 0x7f539c0b5ca0 2024-04-26T19:59:43.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.200+0000 7f5393400640 1 -- 172.21.15.88:0/3009503652 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f5384059d80 con 0x7f539c0b5ca0 2024-04-26T19:59:43.202 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 msgr2=0x7f537806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 0x7f537806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f538c000f60 tx=0x7f538c0079d0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 msgr2=0x7f539c152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f538400d800 tx=0x7f538400dcc0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 shutdown_connections 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f537806c8f0 0x7f537806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f539c15d100 0x7f539c09cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c0b5ca0 0x7f539c152fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 --2- 172.21.15.88:0/3009503652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f539c06ce80 0x7f539c14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 >> 172.21.15.88:0/3009503652 conn(0x7f539c0b2d50 msgr2=0x7f539c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 shutdown_connections 2024-04-26T19:59:43.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.203+0000 7f53a2400640 1 -- 172.21.15.88:0/3009503652 wait complete. 2024-04-26T19:59:43.226 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T19:59:43.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.421+0000 7f1a9e200640 1 Processor -- start 2024-04-26T19:59:43.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.421+0000 7f1a9e200640 1 -- start start 2024-04-26T19:59:43.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.423+0000 7f1a9e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 0x7f1a980cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.423+0000 7f1a9e200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a980cf360 0x7f1a980cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.423+0000 7f1a9e200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a9806dee0 con 0x7f1a980cd010 2024-04-26T19:59:43.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.423+0000 7f1a9e200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1a9806f940 con 0x7f1a980cf360 2024-04-26T19:59:43.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.423+0000 7f1a9e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9806e180 con 0x7f1a981668f0 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 0x7f1a980cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a8fe00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f1a980cd010 0x7f1a980cb670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:44362/0 (socket says 172.21.15.88:44362) 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a8fe00640 1 -- 172.21.15.88:0/1572814505 learned_addr learned my addr 172.21.15.88:0/1572814505 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9d800640 1 --2- 172.21.15.88:0/1572814505 >> v2:172.21.15.132:3300/0 conn(0x7f1a980cf360 0x7f1a980cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a97e00640 1 -- 172.21.15.88:0/1572814505 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191706524 0 0) 0x7f1a9806dee0 con 0x7f1a980cd010 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9ce00640 1 -- 172.21.15.88:0/1572814505 >> v2:172.21.15.132:3300/0 conn(0x7f1a980cf360 msgr2=0x7f1a980cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/1572814505 >> v2:172.21.15.132:3300/0 conn(0x7f1a980cf360 0x7f1a980cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9ce00640 1 -- 172.21.15.88:0/1572814505 >> v1:172.21.15.132:6789/0 conn(0x7f1a980cd010 legacy=0x7f1a980cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.424+0000 7f1a9ce00640 1 -- 172.21.15.88:0/1572814505 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9806fb70 con 0x7f1a981668f0 2024-04-26T19:59:43.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.426+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/1572814505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 0x7f1a980cb130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a880097a0 tx=0x7f1a8802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eb84fe1325c2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.426+0000 7f1a97e00640 1 -- 172.21.15.88:0/1572814505 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88037070 con 0x7f1a981668f0 2024-04-26T19:59:43.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.426+0000 7f1a97e00640 1 -- 172.21.15.88:0/1572814505 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8802ae50 con 0x7f1a981668f0 2024-04-26T19:59:43.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.426+0000 7f1a97e00640 1 -- 172.21.15.88:0/1572814505 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88032660 con 0x7f1a981668f0 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 -- 172.21.15.88:0/1572814505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 msgr2=0x7f1a980cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 --2- 172.21.15.88:0/1572814505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 0x7f1a980cb130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a880097a0 tx=0x7f1a8802a100 comp rx=0 tx=0).stop 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 -- 172.21.15.88:0/1572814505 shutdown_connections 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 --2- 172.21.15.88:0/1572814505 >> v2:172.21.15.132:3300/0 conn(0x7f1a980cf360 0x7f1a980cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 --2- 172.21.15.88:0/1572814505 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a981668f0 0x7f1a980cb130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 -- 172.21.15.88:0/1572814505 >> 172.21.15.88:0/1572814505 conn(0x7f1a980d0ae0 msgr2=0x7f1a980d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 -- 172.21.15.88:0/1572814505 shutdown_connections 2024-04-26T19:59:43.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.427+0000 7f1a9e200640 1 -- 172.21.15.88:0/1572814505 wait complete. 2024-04-26T19:59:43.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.429+0000 7f1a9e200640 1 Processor -- start 2024-04-26T19:59:43.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.429+0000 7f1a9e200640 1 -- start start 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 0x7f1a9815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 0x7f1a981fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98070130 con 0x7f1a980cf360 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98166cd0 con 0x7f1a981668f0 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a9e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9806e460 con 0x7f1a980cd010 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58208/0 (socket says 172.21.15.88:58208) 2024-04-26T19:59:43.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.430+0000 7f1a8fe00640 1 -- 172.21.15.88:0/2860227232 learned_addr learned my addr 172.21.15.88:0/2860227232 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a9d800640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 0x7f1a981fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 0x7f1a9815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 -- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 msgr2=0x7f1a9815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 0x7f1a9815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 -- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 msgr2=0x7f1a981fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 0x7f1a981fbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 -- 172.21.15.88:0/2860227232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a98163710 con 0x7f1a980cf360 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 0x7f1a9815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a9d800640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 0x7f1a981fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a8fe00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000c4a0 tx=0x7f1a8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80018070 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a800074e0 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004a20 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.431+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a981639f0 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.432+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a98163fb0 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.433+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a980693c0 con 0x7f1a980cf360 2024-04-26T19:59:43.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.434+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a80004bc0 con 0x7f1a980cf360 2024-04-26T19:59:43.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.434+0000 7f1a95600640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.435+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1a8008aba0 con 0x7f1a980cf360 2024-04-26T19:59:43.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.435+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.435+0000 7f1a9ce00640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 0x7f1a7406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a88034000 tx=0x7f1a8802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.440+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a80059310 con 0x7f1a980cf360 2024-04-26T19:59:43.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.642+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1a98076600 con 0x7f1a980cf360 2024-04-26T19:59:43.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.643+0000 7f1a95600640 1 -- 172.21.15.88:0/2860227232 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f1a80058c10 con 0x7f1a980cf360 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T19:59:43.645 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T19:59:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 msgr2=0x7f1a7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 0x7f1a7406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a88034000 tx=0x7f1a8802e000 comp rx=0 tx=0).stop 2024-04-26T19:59:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 msgr2=0x7f1a981fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000c4a0 tx=0x7f1a8000c960 comp rx=0 tx=0).stop 2024-04-26T19:59:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 shutdown_connections 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a981668f0 0x7f1a981fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980cf360 0x7f1a981fb5c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a7406c8f0 0x7f1a7406edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 --2- 172.21.15.88:0/2860227232 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a980cd010 0x7f1a9815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.645+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 >> 172.21.15.88:0/2860227232 conn(0x7f1a980d0ae0 msgr2=0x7f1a980cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.646+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 shutdown_connections 2024-04-26T19:59:43.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.646+0000 7f1a9e200640 1 -- 172.21.15.88:0/2860227232 wait complete. 2024-04-26T19:59:43.665 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T19:59:43.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T19:59:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.865+0000 7feefd200640 1 Processor -- start 2024-04-26T19:59:43.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.865+0000 7feefd200640 1 -- start start 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feefd200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feef806f9d0 0x7feef806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feefd200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 0x7feef80b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feefd200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef80b5f40 con 0x7feef815bf50 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feefd200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7feef8070370 con 0x7feef806f9d0 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feefd200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7feef815c330 con 0x7feef80b5210 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feef5400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feef806f9d0 0x7feef806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feef5400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feef806f9d0 0x7feef806fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58224/0 (socket says 172.21.15.88:58224) 2024-04-26T19:59:43.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feef5400640 1 -- 172.21.15.88:0/1678670488 learned_addr learned my addr 172.21.15.88:0/1678670488 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feef6800640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 0x7feef80b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.867+0000 7feef5400640 1 -- 172.21.15.88:0/1678670488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 msgr2=0x7feef80b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.868+0000 7feef5400640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 0x7feef80b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.868+0000 7feef5400640 1 -- 172.21.15.88:0/1678670488 >> v1:172.21.15.132:6789/0 conn(0x7feef815bf50 legacy=0x7feef8072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.868+0000 7feef5400640 1 -- 172.21.15.88:0/1678670488 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80aeaa0 con 0x7feef806f9d0 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.868+0000 7feef6800640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 0x7feef80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:43.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.869+0000 7feef5400640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef806fe30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feee4009870 tx=0x7feee402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a5eb0df5417539 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.869+0000 7feef4a00640 1 -- 172.21.15.88:0/1678670488 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4037070 con 0x7feef806f9d0 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.869+0000 7feef4a00640 1 -- 172.21.15.88:0/1678670488 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee4032440 con 0x7feef806f9d0 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.869+0000 7feef4a00640 1 -- 172.21.15.88:0/1678670488 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4032810 con 0x7feef806f9d0 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 -- 172.21.15.88:0/1678670488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 msgr2=0x7feef806fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef806fe30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feee4009870 tx=0x7feee402a210 comp rx=0 tx=0).stop 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 -- 172.21.15.88:0/1678670488 shutdown_connections 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef80b5210 0x7feef80b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 --2- 172.21.15.88:0/1678670488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef806fe30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 -- 172.21.15.88:0/1678670488 >> 172.21.15.88:0/1678670488 conn(0x7feef80d0c40 msgr2=0x7feef80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 -- 172.21.15.88:0/1678670488 shutdown_connections 2024-04-26T19:59:43.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.870+0000 7feefd200640 1 -- 172.21.15.88:0/1678670488 wait complete. 2024-04-26T19:59:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.871+0000 7feefd200640 1 Processor -- start 2024-04-26T19:59:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.871+0000 7feefd200640 1 -- start start 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef80b5210 0x7feef815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef815bf50 0x7feef815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feef80af340 con 0x7feef806f9d0 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feef80af060 con 0x7feef80b5210 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.872+0000 7feefd200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7feef80b5be0 con 0x7feef815bf50 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58234/0 (socket says 172.21.15.88:58234) 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 -- 172.21.15.88:0/405720526 learned_addr learned my addr 172.21.15.88:0/405720526 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 -- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef815bf50 msgr2=0x7feef815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef815bf50 0x7feef815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 -- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef80b5210 msgr2=0x7feef815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef80b5210 0x7feef815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 -- 172.21.15.88:0/405720526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef815c750 con 0x7feef806f9d0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feef5e00640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feee800b3f0 tx=0x7feee800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.873+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8015020 con 0x7feef806f9d0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.874+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8011830 con 0x7feef806f9d0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.874+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8011c20 con 0x7feef806f9d0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.874+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef815ca30 con 0x7feef806f9d0 2024-04-26T19:59:43.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.874+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef815cff0 con 0x7feef806f9d0 2024-04-26T19:59:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.875+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef8069650 con 0x7feef806f9d0 2024-04-26T19:59:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.876+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feee8018020 con 0x7feef806f9d0 2024-04-26T19:59:43.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.877+0000 7feeee000640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 0x7feed806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:43.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.877+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7feee808b370 con 0x7feef806f9d0 2024-04-26T19:59:43.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.878+0000 7feef5400640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 0x7feed806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:43.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.878+0000 7feef5400640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 0x7feed806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feee4034000 tx=0x7feee402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:43.882+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feee801f030 con 0x7feef806f9d0 2024-04-26T19:59:44.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.085+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feef806e340 con 0x7feef806f9d0 2024-04-26T19:59:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.236+0000 7feeee000640 1 -- 172.21.15.88:0/405720526 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7feee80172c0 con 0x7feef806f9d0 2024-04-26T19:59:44.238 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-04-26T19:59:44.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.239+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 msgr2=0x7feed806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.239+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 0x7feed806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feee4034000 tx=0x7feee402c000 comp rx=0 tx=0).stop 2024-04-26T19:59:44.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.239+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 msgr2=0x7feef809d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.239+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feee800b3f0 tx=0x7feee800b8b0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 shutdown_connections 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feed806c920 0x7feed806ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feef815bf50 0x7feef815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef80b5210 0x7feef815e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 --2- 172.21.15.88:0/405720526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef806f9d0 0x7feef809d870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 >> 172.21.15.88:0/405720526 conn(0x7feef80d0c40 msgr2=0x7feef806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 shutdown_connections 2024-04-26T19:59:44.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.240+0000 7feefd200640 1 -- 172.21.15.88:0/405720526 wait complete. 2024-04-26T19:59:44.272 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T19:59:44.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.463+0000 7fd7a1000640 1 Processor -- start 2024-04-26T19:59:44.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.463+0000 7fd7a1000640 1 -- start start 2024-04-26T19:59:44.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd7a1000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd79c0b5390 0x7fd79c06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd7a1000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c06eb10 con 0x7fd79c1677f0 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd7a1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c06edf0 con 0x7fd79c15d780 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd7a1000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c06f0d0 con 0x7fd79c0b5390 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd799400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd79c1677f0 0x7fd79c06dde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45228/0 (socket says 172.21.15.88:45228) 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd799400640 1 -- 172.21.15.88:0/3323750697 learned_addr learned my addr 172.21.15.88:0/3323750697 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.465+0000 7fd79a800640 1 --2- 172.21.15.88:0/3323750697 >> v2:172.21.15.88:3300/0 conn(0x7fd79c0b5390 0x7fd79c06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:44.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.466+0000 7fd798a00640 1 -- 172.21.15.88:0/3323750697 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158287459 0 0) 0x7fd79c06eb10 con 0x7fd79c1677f0 2024-04-26T19:59:44.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.466+0000 7fd798a00640 1 -- 172.21.15.88:0/3323750697 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd780003520 con 0x7fd79c1677f0 2024-04-26T19:59:44.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.466+0000 7fd79a800640 1 -- 172.21.15.88:0/3323750697 >> v1:172.21.15.132:6789/0 conn(0x7fd79c15d780 legacy=0x7fd79c1670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.466+0000 7fd79a800640 1 -- 172.21.15.88:0/3323750697 >> v1:172.21.15.88:6789/0 conn(0x7fd79c1677f0 legacy=0x7fd79c06dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.466+0000 7fd79a800640 1 -- 172.21.15.88:0/3323750697 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c06f3b0 con 0x7fd79c0b5390 2024-04-26T19:59:44.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.467+0000 7fd79a800640 1 --2- 172.21.15.88:0/3323750697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c0b5390 0x7fd79c06e4f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e756e0895406c208 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:44.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.467+0000 7fd798a00640 1 -- 172.21.15.88:0/3323750697 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794007740 con 0x7fd79c0b5390 2024-04-26T19:59:44.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.467+0000 7fd798a00640 1 -- 172.21.15.88:0/3323750697 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7940078e0 con 0x7fd79c0b5390 2024-04-26T19:59:44.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.467+0000 7fd798a00640 1 -- 172.21.15.88:0/3323750697 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794007cb0 con 0x7fd79c0b5390 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 -- 172.21.15.88:0/3323750697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c0b5390 msgr2=0x7fd79c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 --2- 172.21.15.88:0/3323750697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c0b5390 0x7fd79c06e4f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402a6f0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 -- 172.21.15.88:0/3323750697 shutdown_connections 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 --2- 172.21.15.88:0/3323750697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c0b5390 0x7fd79c06e4f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 -- 172.21.15.88:0/3323750697 >> 172.21.15.88:0/3323750697 conn(0x7fd79c0b2a30 msgr2=0x7fd79c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 -- 172.21.15.88:0/3323750697 shutdown_connections 2024-04-26T19:59:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.468+0000 7fd7a1000640 1 -- 172.21.15.88:0/3323750697 wait complete. 2024-04-26T19:59:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.470+0000 7fd7a1000640 1 Processor -- start 2024-04-26T19:59:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.470+0000 7fd7a1000640 1 -- start start 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79c0b5390 0x7fd79c09bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd79c1677f0 0x7fd79c1eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c15db60 con 0x7fd79c15d780 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0728c0 con 0x7fd79c1677f0 2024-04-26T19:59:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd7a1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c15dce0 con 0x7fd79c0b5390 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd799400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd799400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58248/0 (socket says 172.21.15.88:58248) 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.471+0000 7fd799400640 1 -- 172.21.15.88:0/2079991308 learned_addr learned my addr 172.21.15.88:0/2079991308 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799e00640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79c0b5390 0x7fd79c09bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd79a800640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd79c1677f0 0x7fd79c1eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 -- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79c0b5390 msgr2=0x7fd79c09bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79c0b5390 0x7fd79c09bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 -- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd79c1677f0 msgr2=0x7fd79c1eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd79c1677f0 0x7fd79c1eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 -- 172.21.15.88:0/2079991308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1564d0 con 0x7fd79c15d780 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd799400640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00c440 tx=0x7fd78c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017070 con 0x7fd79c15d780 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c005ce0 con 0x7fd79c15d780 2024-04-26T19:59:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0072b0 con 0x7fd79c15d780 2024-04-26T19:59:44.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.472+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1ef130 con 0x7fd79c15d780 2024-04-26T19:59:44.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.473+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1575b0 con 0x7fd79c15d780 2024-04-26T19:59:44.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.474+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c069650 con 0x7fd79c15d780 2024-04-26T19:59:44.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.475+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd78c01a070 con 0x7fd79c15d780 2024-04-26T19:59:44.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.476+0000 7fd792000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 0x7fd77806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:44.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.476+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fd78c08ad80 con 0x7fd79c15d780 2024-04-26T19:59:44.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.476+0000 7fd799e00640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 0x7fd77806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:44.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.477+0000 7fd799e00640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 0x7fd77806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd784000f60 tx=0x7fd784009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:44.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.481+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd78c0593b0 con 0x7fd79c15d780 2024-04-26T19:59:44.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:44.682+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd79c1ef650 con 0x7fd79c15d780 2024-04-26T19:59:45.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.250+0000 7fd792000640 1 -- 172.21.15.88:0/2079991308 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fd78c058cb0 con 0x7fd79c15d780 2024-04-26T19:59:45.252 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-04-26T19:59:45.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.254+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 msgr2=0x7fd77806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:45.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.254+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 0x7fd77806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd784000f60 tx=0x7fd784009040 comp rx=0 tx=0).stop 2024-04-26T19:59:45.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.254+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 msgr2=0x7fd79c1ee5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:45.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.254+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00c440 tx=0x7fd78c00c900 comp rx=0 tx=0).stop 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 shutdown_connections 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd79c1677f0 0x7fd79c1eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd79c15d780 0x7fd79c1ee5e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00c440 tx=0x7fd78c00c900 comp rx=0 tx=0).stop 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd77806c8f0 0x7fd77806edb0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd784000f60 tx=0x7fd784009040 comp rx=0 tx=0).stop 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 --2- 172.21.15.88:0/2079991308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd79c0b5390 0x7fd79c09bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.255+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 >> 172.21.15.88:0/2079991308 conn(0x7fd79c0b2a30 msgr2=0x7fd79c0aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:45.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.256+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 shutdown_connections 2024-04-26T19:59:45.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.256+0000 7fd7a1000640 1 -- 172.21.15.88:0/2079991308 wait complete. 2024-04-26T19:59:45.292 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T19:59:45.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.465+0000 7f98b9000640 1 Processor -- start 2024-04-26T19:59:45.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.465+0000 7f98b9000640 1 -- start start 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b415d100 0x7f98b406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f98b40b5ca0 0x7f98b406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f98b40add80 0x7f98b4072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40af800 con 0x7f98b40add80 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40ae340 con 0x7f98b40b5ca0 2024-04-26T19:59:45.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b9000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f98b40afaa0 con 0x7f98b415d100 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b2800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f98b40add80 0x7f98b4072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b2800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f98b40add80 0x7f98b4072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58252/0 (socket says 172.21.15.88:58252) 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b2800640 1 -- 172.21.15.88:0/2621484632 learned_addr learned my addr 172.21.15.88:0/2621484632 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.467+0000 7f98b1e00640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b415d100 0x7f98b406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98abe00640 1 --2- 172.21.15.88:0/2621484632 >> v2:172.21.15.132:3300/0 conn(0x7f98b40b5ca0 0x7f98b406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98b2800640 1 -- 172.21.15.88:0/2621484632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b415d100 msgr2=0x7f98b406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98b2800640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b415d100 0x7f98b406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98b2800640 1 -- 172.21.15.88:0/2621484632 >> v2:172.21.15.132:3300/0 conn(0x7f98b40b5ca0 msgr2=0x7f98b406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98b2800640 1 --2- 172.21.15.88:0/2621484632 >> v2:172.21.15.132:3300/0 conn(0x7f98b40b5ca0 0x7f98b406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.468+0000 7f98b2800640 1 -- 172.21.15.88:0/2621484632 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40afd80 con 0x7f98b40add80 2024-04-26T19:59:45.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.469+0000 7f98b2800640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40add80 0x7f98b4072010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98b4075d50 tx=0x7f98ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffe7289942b335e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:45.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.469+0000 7f98b1400640 1 -- 172.21.15.88:0/2621484632 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac036070 con 0x7f98b40add80 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.469+0000 7f98b1400640 1 -- 172.21.15.88:0/2621484632 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac031600 con 0x7f98b40add80 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.469+0000 7f98b1400640 1 -- 172.21.15.88:0/2621484632 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0319d0 con 0x7f98b40add80 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 -- 172.21.15.88:0/2621484632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40add80 msgr2=0x7f98b4072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40add80 0x7f98b4072010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98b4075d50 tx=0x7f98ac02a610 comp rx=0 tx=0).stop 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 -- 172.21.15.88:0/2621484632 shutdown_connections 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40add80 0x7f98b4072010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 --2- 172.21.15.88:0/2621484632 >> v2:172.21.15.132:3300/0 conn(0x7f98b40b5ca0 0x7f98b406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 --2- 172.21.15.88:0/2621484632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b415d100 0x7f98b406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 -- 172.21.15.88:0/2621484632 >> 172.21.15.88:0/2621484632 conn(0x7f98b40b2d50 msgr2=0x7f98b40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 -- 172.21.15.88:0/2621484632 shutdown_connections 2024-04-26T19:59:45.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.470+0000 7f98b9000640 1 -- 172.21.15.88:0/2621484632 wait complete. 2024-04-26T19:59:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.472+0000 7f98b9000640 1 Processor -- start 2024-04-26T19:59:45.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.472+0000 7f98b9000640 1 -- start start 2024-04-26T19:59:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b40add80 0x7f98b4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 0x7f98b414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98b409ee50 con 0x7f98b40b5ca0 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40b0340 con 0x7f98b415d100 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b9000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f98b4160b70 con 0x7f98b40add80 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58266/0 (socket says 172.21.15.88:58266) 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 -- 172.21.15.88:0/1460352386 learned_addr learned my addr 172.21.15.88:0/1460352386 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b1e00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b40add80 0x7f98b4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b2800640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 0x7f98b414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 -- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b40add80 msgr2=0x7f98b4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b40add80 0x7f98b4151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 -- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 msgr2=0x7f98b414f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 0x7f98b414f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98abe00640 1 -- 172.21.15.88:0/1460352386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b415ec50 con 0x7f98b40b5ca0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.473+0000 7f98b2800640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 0x7f98b414f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98abe00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d830 tx=0x7f989c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c00b6c0 con 0x7f98b40b5ca0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989c019070 con 0x7f98b40b5ca0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c00ba80 con 0x7f98b40b5ca0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b414f930 con 0x7f98b40b5ca0 2024-04-26T19:59:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.474+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b414fc10 con 0x7f98b40b5ca0 2024-04-26T19:59:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.475+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b4069650 con 0x7f98b40b5ca0 2024-04-26T19:59:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.476+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f989c01e460 con 0x7f98b40b5ca0 2024-04-26T19:59:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.477+0000 7f98aa000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 0x7f989006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:45.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.477+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f989c08c110 con 0x7f98b40b5ca0 2024-04-26T19:59:45.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.477+0000 7f98b1e00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 0x7f989006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:45.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.478+0000 7f98b1e00640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 0x7f989006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009a20 tx=0x7f98a4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:45.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.482+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f989c05a590 con 0x7f98b40b5ca0 2024-04-26T19:59:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:45.719+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f98b4073d00 con 0x7f98b40b5ca0 2024-04-26T19:59:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.254+0000 7f98aa000640 1 -- 172.21.15.88:0/1460352386 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f989c059e90 con 0x7f98b40b5ca0 2024-04-26T19:59:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T19:59:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.258+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 msgr2=0x7f989006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.258+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 0x7f989006ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009a20 tx=0x7f98a4009190 comp rx=0 tx=0).stop 2024-04-26T19:59:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.258+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 msgr2=0x7f98b41516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.258+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d830 tx=0x7f989c00dcf0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 shutdown_connections 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98b415d100 0x7f98b414f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98b40b5ca0 0x7f98b41516d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f989006c920 0x7f989006ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 --2- 172.21.15.88:0/1460352386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f98b40add80 0x7f98b4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.259+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 >> 172.21.15.88:0/1460352386 conn(0x7f98b40b2d50 msgr2=0x7f98b4158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:46.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.261+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 shutdown_connections 2024-04-26T19:59:46.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.263+0000 7f98b9000640 1 -- 172.21.15.88:0/1460352386 wait complete. 2024-04-26T19:59:46.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:46.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.475+0000 7f0046c00640 1 Processor -- start 2024-04-26T19:59:46.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.476+0000 7f0046c00640 1 -- start start 2024-04-26T19:59:46.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f004015f1b0 0x7f00400d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f0040162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0040163910 con 0x7f0040163480 2024-04-26T19:59:46.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f00400d3f50 con 0x7f004015f1b0 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f00400d36b0 con 0x7f0040162b00 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0045800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f004015f1b0 0x7f00400d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0040163480 0x7f00400cfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45240/0 (socket says 172.21.15.88:45240) 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0046200640 1 -- 172.21.15.88:0/2236007991 learned_addr learned my addr 172.21.15.88:0/2236007991 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.477+0000 7f0045800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f004015f1b0 0x7f00400d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58276/0 (socket says 172.21.15.88:58276) 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f0044e00640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f0040162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f0045800640 1 -- 172.21.15.88:0/2236007991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 msgr2=0x7f0040162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f0045800640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f0040162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f0045800640 1 -- 172.21.15.88:0/2236007991 >> v1:172.21.15.88:6789/0 conn(0x7f0040163480 legacy=0x7f00400cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f0045800640 1 -- 172.21.15.88:0/2236007991 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00400d3990 con 0x7f004015f1b0 2024-04-26T19:59:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.478+0000 7f003fe00640 1 -- 172.21.15.88:0/2236007991 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609293683 0 0) 0x7f0040163910 con 0x7f0040163480 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.479+0000 7f0045800640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f004015f1b0 0x7f00400d5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0034009d90 tx=0x7f003402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea6bc81aed2efc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.479+0000 7f003fe00640 1 -- 172.21.15.88:0/2236007991 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034007740 con 0x7f004015f1b0 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.479+0000 7f003fe00640 1 -- 172.21.15.88:0/2236007991 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00340078e0 con 0x7f004015f1b0 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.479+0000 7f003fe00640 1 -- 172.21.15.88:0/2236007991 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034007cb0 con 0x7f004015f1b0 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 -- 172.21.15.88:0/2236007991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f004015f1b0 msgr2=0x7f00400d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f004015f1b0 0x7f00400d5990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0034009d90 tx=0x7f003402a660 comp rx=0 tx=0).stop 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 -- 172.21.15.88:0/2236007991 shutdown_connections 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f0040162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 --2- 172.21.15.88:0/2236007991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f004015f1b0 0x7f00400d5990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 -- 172.21.15.88:0/2236007991 >> 172.21.15.88:0/2236007991 conn(0x7f004009e230 msgr2=0x7f00400d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 -- 172.21.15.88:0/2236007991 shutdown_connections 2024-04-26T19:59:46.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.480+0000 7f0046c00640 1 -- 172.21.15.88:0/2236007991 wait complete. 2024-04-26T19:59:46.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.482+0000 7f0046c00640 1 Processor -- start 2024-04-26T19:59:46.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.482+0000 7f0046c00640 1 -- start start 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f004015f1b0 0x7f0040154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f00400c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00400d33a0 con 0x7f0040163480 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f00400b7d90 con 0x7f004015f1b0 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f00400b92c0 con 0x7f0040162b00 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58288/0 (socket says 172.21.15.88:58288) 2024-04-26T19:59:46.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046200640 1 -- 172.21.15.88:0/1915017363 learned_addr learned my addr 172.21.15.88:0/1915017363 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.483+0000 7f0046200640 1 -- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 msgr2=0x7f00400c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046200640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f00400c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046200640 1 -- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f004015f1b0 msgr2=0x7f0040154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0045800640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f004015f1b0 0x7f0040154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046200640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f004015f1b0 0x7f0040154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046200640 1 -- 172.21.15.88:0/1915017363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00400cccb0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046200640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003800c410 tx=0x7f003800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00380074e0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00380047e0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038004bd0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00401601e0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.484+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00401607a0 con 0x7f0040163480 2024-04-26T19:59:46.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.485+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00401609a0 con 0x7f0040163480 2024-04-26T19:59:46.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.486+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0038011070 con 0x7f0040163480 2024-04-26T19:59:46.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.487+0000 7f003d600640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 0x7f002006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.487+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f003808b6a0 con 0x7f0040163480 2024-04-26T19:59:46.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.487+0000 7f0045800640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 0x7f002006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.488+0000 7f0045800640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 0x7f002006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0034034000 tx=0x7f003402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.492+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00401609a0 con 0x7f0040163480 2024-04-26T19:59:46.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.687+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00400693c0 con 0x7f0040163480 2024-04-26T19:59:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.689+0000 7f003d600640 1 -- 172.21.15.88:0/1915017363 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16606 (secure 0 0 0) 0x7f003801a2b0 con 0x7f0040163480 2024-04-26T19:59:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:46.691 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:45.722127+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 msgr2=0x7f002006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 0x7f002006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0034034000 tx=0x7f003402f000 comp rx=0 tx=0).stop 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 msgr2=0x7f004015d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f003800c410 tx=0x7f003800c8d0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 shutdown_connections 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0040163480 0x7f004015d530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f002006c8f0 0x7f002006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0040162b00 0x7f00400c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 --2- 172.21.15.88:0/1915017363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f004015f1b0 0x7f0040154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 >> 172.21.15.88:0/1915017363 conn(0x7f004009e230 msgr2=0x7f004006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 shutdown_connections 2024-04-26T19:59:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.691+0000 7f0046c00640 1 -- 172.21.15.88:0/1915017363 wait complete. 2024-04-26T19:59:46.710 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.888+0000 7fd2f3e00640 1 Processor -- start 2024-04-26T19:59:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.888+0000 7fd2f3e00640 1 -- start start 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd2f415f1b0 0x7fd2f40d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd2f4163650 0x7fd2f40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2f40d3f50 con 0x7fd2f4162b00 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f3e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd2f40d36b0 con 0x7fd2f415f1b0 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd2f40d3990 con 0x7fd2f4163650 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd2f415f1b0 0x7fd2f40d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd2f415f1b0 0x7fd2f40d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58292/0 (socket says 172.21.15.88:58292) 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 -- 172.21.15.88:0/1692629620 learned_addr learned my addr 172.21.15.88:0/1692629620 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 -- 172.21.15.88:0/1692629620 >> v2:172.21.15.132:3300/0 conn(0x7fd2f4163650 msgr2=0x7fd2f40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:46.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 --2- 172.21.15.88:0/1692629620 >> v2:172.21.15.132:3300/0 conn(0x7fd2f4163650 0x7fd2f40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 -- 172.21.15.88:0/1692629620 >> v1:172.21.15.132:6789/0 conn(0x7fd2f4162b00 legacy=0x7fd2f4162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.890+0000 7fd2f2a00640 1 -- 172.21.15.88:0/1692629620 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f40d3c70 con 0x7fd2f415f1b0 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.891+0000 7fd2f2a00640 1 --2- 172.21.15.88:0/1692629620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f415f1b0 0x7fd2f40d5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2f415f590 tx=0x7fd2e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee608ffb519b17da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f2000640 1 -- 172.21.15.88:0/1692629620 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4037070 con 0x7fd2f415f1b0 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f2000640 1 -- 172.21.15.88:0/1692629620 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e4032440 con 0x7fd2f415f1b0 2024-04-26T19:59:46.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f2000640 1 -- 172.21.15.88:0/1692629620 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4032810 con 0x7fd2f415f1b0 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 -- 172.21.15.88:0/1692629620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f415f1b0 msgr2=0x7fd2f40d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/1692629620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f415f1b0 0x7fd2f40d5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2f415f590 tx=0x7fd2e402a230 comp rx=0 tx=0).stop 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 -- 172.21.15.88:0/1692629620 shutdown_connections 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/1692629620 >> v2:172.21.15.132:3300/0 conn(0x7fd2f4163650 0x7fd2f40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/1692629620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f415f1b0 0x7fd2f40d5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 -- 172.21.15.88:0/1692629620 >> 172.21.15.88:0/1692629620 conn(0x7fd2f409e230 msgr2=0x7fd2f40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:46.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 -- 172.21.15.88:0/1692629620 shutdown_connections 2024-04-26T19:59:46.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.892+0000 7fd2f3e00640 1 -- 172.21.15.88:0/1692629620 wait complete. 2024-04-26T19:59:46.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.894+0000 7fd2f3e00640 1 Processor -- start 2024-04-26T19:59:46.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.894+0000 7fd2f3e00640 1 -- start start 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f415f1b0 0x7fd2f415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2f4162b00 0x7fd2f40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f40b87f0 con 0x7fd2f4163650 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f40d33a0 con 0x7fd2f415f1b0 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.895+0000 7fd2f3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f40b8970 con 0x7fd2f4162b00 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58300/0 (socket says 172.21.15.88:58300) 2024-04-26T19:59:46.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 -- 172.21.15.88:0/605026459 learned_addr learned my addr 172.21.15.88:0/605026459 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2ea000640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2f4162b00 0x7fd2f40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 -- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2f4162b00 msgr2=0x7fd2f40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2f4162b00 0x7fd2f40cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 -- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f415f1b0 msgr2=0x7fd2f415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f415f1b0 0x7fd2f415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 -- 172.21.15.88:0/605026459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f415e410 con 0x7fd2f4163650 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2f3400640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b9e0 tx=0x7fd2ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.896+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec019070 con 0x7fd2f4163650 2024-04-26T19:59:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.897+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ec00cb20 con 0x7fd2f4163650 2024-04-26T19:59:46.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.897+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec014440 con 0x7fd2f4163650 2024-04-26T19:59:46.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.897+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f40ce900 con 0x7fd2f4163650 2024-04-26T19:59:46.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.897+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f41e9c00 con 0x7fd2f4163650 2024-04-26T19:59:46.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.898+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f40693c0 con 0x7fd2f4163650 2024-04-26T19:59:46.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.899+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2ec01c070 con 0x7fd2f4163650 2024-04-26T19:59:46.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.900+0000 7fd2eb400640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 0x7fd2d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:46.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.900+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fd2ec08b8a0 con 0x7fd2f4163650 2024-04-26T19:59:46.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.900+0000 7fd2f2a00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 0x7fd2d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:46.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.901+0000 7fd2f2a00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 0x7fd2d006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4034000 tx=0x7fd2e402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:46.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:46.905+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2ec059cb0 con 0x7fd2f4163650 2024-04-26T19:59:47.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.090+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2f408d090 con 0x7fd2f4163650 2024-04-26T19:59:47.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.093+0000 7fd2eb400640 1 -- 172.21.15.88:0/605026459 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16606 (secure 0 0 0) 0x7fd2ec0595b0 con 0x7fd2f4163650 2024-04-26T19:59:47.095 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:47.095 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:45.722127+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.095+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 msgr2=0x7fd2d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.095+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 0x7fd2d006edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4034000 tx=0x7fd2e402c000 comp rx=0 tx=0).stop 2024-04-26T19:59:47.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.095+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 msgr2=0x7fd2f40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.095+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b9e0 tx=0x7fd2ec00bea0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 shutdown_connections 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4163650 0x7fd2f40ce2a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2d006c8f0 0x7fd2d006edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2f4162b00 0x7fd2f40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 --2- 172.21.15.88:0/605026459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f415f1b0 0x7fd2f415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 >> 172.21.15.88:0/605026459 conn(0x7fd2f409e230 msgr2=0x7fd2f40b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 shutdown_connections 2024-04-26T19:59:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.096+0000 7fd2f3e00640 1 -- 172.21.15.88:0/605026459 wait complete. 2024-04-26T19:59:47.115 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T19:59:47.229 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-04-26T19:59:47.227+0000 7f32f9600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T19:59:47.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.315+0000 7f5f83a00640 1 Processor -- start 2024-04-26T19:59:47.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.315+0000 7f5f83a00640 1 -- start start 2024-04-26T19:59:47.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f83a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5f7c147f20 0x7f5f7c0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f83a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7c0ade20 con 0x7f5f7c06ce80 2024-04-26T19:59:47.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f83a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f7c0ae100 con 0x7f5f7c073da0 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f83a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f5f7c0ae3e0 con 0x7f5f7c147f20 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f81a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5f7c06ce80 0x7f5f7c0aeec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:45244/0 (socket says 172.21.15.88:45244) 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f81a00640 1 -- 172.21.15.88:0/763410959 learned_addr learned my addr 172.21.15.88:0/763410959 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347641841 0 0) 0x7f5f7c0ade20 con 0x7f5f7c06ce80 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.317+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f64003520 con 0x7f5f7c06ce80 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f81000640 1 --2- 172.21.15.88:0/763410959 >> v2:172.21.15.132:3300/0 conn(0x7f5f7c147f20 0x7f5f7c0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 562886286 0 0) 0x7f5f64003520 con 0x7f5f7c06ce80 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 >> v2:172.21.15.132:3300/0 conn(0x7f5f7c147f20 msgr2=0x7f5f7c0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 --2- 172.21.15.88:0/763410959 >> v2:172.21.15.132:3300/0 conn(0x7f5f7c147f20 0x7f5f7c0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 >> v1:172.21.15.132:6789/0 conn(0x7f5f7c073da0 legacy=0x7f5f7c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c0ae6c0 con 0x7f5f7c06ce80 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f5f740019f0 con 0x7f5f7c06ce80 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 >> v1:172.21.15.88:6789/0 conn(0x7f5f7c06ce80 legacy=0x7f5f7c0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f640053c0 0x7f5f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c0ae3e0 con 0x7f5f64003a00 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6400c230 con 0x7f5f640053c0 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6400c430 con 0x7f5f64004800 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.318+0000 7f5f81000640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f7be00640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81a00640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f640053c0 0x7f5f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81000640 1 -- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f64004800 msgr2=0x7f5f64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:47.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81000640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f64004800 0x7f5f64004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81000640 1 -- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f640053c0 msgr2=0x7f5f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:47.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81000640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f640053c0 0x7f5f6400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81000640 1 -- 172.21.15.88:0/763410959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f64003520 con 0x7f5f64003a00 2024-04-26T19:59:47.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.319+0000 7f5f81a00640 1 -- 172.21.15.88:0/763410959 reap_dead start 2024-04-26T19:59:47.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.320+0000 7f5f81000640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000a9a0 tx=0x7f5f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0924c3c678b3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:47.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.320+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70037070 con 0x7f5f64003a00 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.320+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7002ab70 con 0x7f5f64003a00 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.320+0000 7f5f7b400640 1 -- 172.21.15.88:0/763410959 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000a570 con 0x7f5f64003a00 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 -- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 msgr2=0x7f5f64003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000a9a0 tx=0x7f5f64005830 comp rx=0 tx=0).stop 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 -- 172.21.15.88:0/763410959 shutdown_connections 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 --2- 172.21.15.88:0/763410959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f64003a00 0x7f5f64003e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 -- 172.21.15.88:0/763410959 >> 172.21.15.88:0/763410959 conn(0x7f5f7c0b2d50 msgr2=0x7f5f7c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 -- 172.21.15.88:0/763410959 shutdown_connections 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.321+0000 7f5f83a00640 1 -- 172.21.15.88:0/763410959 wait complete. 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.322+0000 7f5f83a00640 1 Processor -- start 2024-04-26T19:59:47.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.323+0000 7f5f83a00640 1 -- start start 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.323+0000 7f5f83a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f64003a00 0x7f5f7c160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f83a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f83a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f7c0cebb0 0x7f5f7c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f83a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c06e220 con 0x7f5f7c15ee20 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f83a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c074230 con 0x7f5f64003a00 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f83a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7c0b5c20 con 0x7f5f7c0cebb0 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58320/0 (socket says 172.21.15.88:58320) 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 -- 172.21.15.88:0/1759959737 learned_addr learned my addr 172.21.15.88:0/1759959737 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f81000640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f64003a00 0x7f5f7c160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f81a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f7c0cebb0 0x7f5f7c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 -- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f7c0cebb0 msgr2=0x7f5f7c0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f7c0cebb0 0x7f5f7c0cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 -- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f64003a00 msgr2=0x7f5f7c160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f64003a00 0x7f5f7c160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.324+0000 7f5f7be00640 1 -- 172.21.15.88:0/1759959737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c1620f0 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f7be00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d400 tx=0x7f5f6c00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c018070 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003610 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c161050 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0039e0 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.325+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c161610 con 0x7f5f7c15ee20 2024-04-26T19:59:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.326+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7c069650 con 0x7f5f7c15ee20 2024-04-26T19:59:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.327+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f6c01b070 con 0x7f5f7c15ee20 2024-04-26T19:59:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.328+0000 7f5f78c00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 0x7f5f6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.328+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5f6c08b270 con 0x7f5f7c15ee20 2024-04-26T19:59:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.328+0000 7f5f81000640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 0x7f5f6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.329+0000 7f5f81000640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 0x7f5f6006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f70034000 tx=0x7f5f7002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:47.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.333+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5f6c058fb0 con 0x7f5f7c15ee20 2024-04-26T19:59:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:47.567+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5f7c0738b0 con 0x7f5f7c15ee20 2024-04-26T19:59:48.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.291+0000 7f5f78c00640 1 -- 172.21.15.88:0/1759959737 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f5f6c05d2b0 con 0x7f5f7c15ee20 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.294+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 msgr2=0x7f5f6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 0x7f5f6006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f70034000 tx=0x7f5f7002e000 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 msgr2=0x7f5f7c0caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d400 tx=0x7f5f6c00d8c0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 shutdown_connections 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5f6006c8f0 0x7f5f6006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5f7c0cebb0 0x7f5f7c0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f7c15ee20 0x7f5f7c0caf90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 --2- 172.21.15.88:0/1759959737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f64003a00 0x7f5f7c160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 >> 172.21.15.88:0/1759959737 conn(0x7f5f7c0b2d50 msgr2=0x7f5f7c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 shutdown_connections 2024-04-26T19:59:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.295+0000 7f5f83a00640 1 -- 172.21.15.88:0/1759959737 wait complete. 2024-04-26T19:59:48.318 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T19:59:48.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T19:59:48.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.519+0000 7fbec6c00640 1 Processor -- start 2024-04-26T19:59:48.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.519+0000 7fbec6c00640 1 -- start start 2024-04-26T19:59:48.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.520+0000 7fbec6c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0161140 0x7fbec006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbec0146280 0x7fbec01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00b4d20 con 0x7fbec0160d60 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fbec0146c30 con 0x7fbec0146280 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00b5000 con 0x7fbec0161140 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbec0146280 0x7fbec01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbec0146280 0x7fbec01466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58324/0 (socket says 172.21.15.88:58324) 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 -- 172.21.15.88:0/1156527772 learned_addr learned my addr 172.21.15.88:0/1156527772 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:48.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec4e00640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0161140 0x7fbec006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 -- 172.21.15.88:0/1156527772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0161140 msgr2=0x7fbec006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:48.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0161140 0x7fbec006dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 -- 172.21.15.88:0/1156527772 >> v1:172.21.15.132:6789/0 conn(0x7fbec0160d60 legacy=0x7fbec00b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.521+0000 7fbec6200640 1 -- 172.21.15.88:0/1156527772 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00b52e0 con 0x7fbec0146280 2024-04-26T19:59:48.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.522+0000 7fbec6200640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0146280 0x7fbec01466f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaad145a6775f484 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:48.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbebfe00640 1 -- 172.21.15.88:0/1156527772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8037070 con 0x7fbec0146280 2024-04-26T19:59:48.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbebfe00640 1 -- 172.21.15.88:0/1156527772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb8032440 con 0x7fbec0146280 2024-04-26T19:59:48.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbebfe00640 1 -- 172.21.15.88:0/1156527772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8032810 con 0x7fbec0146280 2024-04-26T19:59:48.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 -- 172.21.15.88:0/1156527772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0146280 msgr2=0x7fbec01466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0146280 0x7fbec01466f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb802a270 comp rx=0 tx=0).stop 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 -- 172.21.15.88:0/1156527772 shutdown_connections 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0146280 0x7fbec01466f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 --2- 172.21.15.88:0/1156527772 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0161140 0x7fbec006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.523+0000 7fbec6c00640 1 -- 172.21.15.88:0/1156527772 >> 172.21.15.88:0/1156527772 conn(0x7fbec00b2900 msgr2=0x7fbec00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.524+0000 7fbec6c00640 1 -- 172.21.15.88:0/1156527772 shutdown_connections 2024-04-26T19:59:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.524+0000 7fbec6c00640 1 -- 172.21.15.88:0/1156527772 wait complete. 2024-04-26T19:59:48.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.525+0000 7fbec6c00640 1 Processor -- start 2024-04-26T19:59:48.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.525+0000 7fbec6c00640 1 -- start start 2024-04-26T19:59:48.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0146280 0x7fbec01f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbec0160d60 0x7fbec0141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006e420 con 0x7fbec0161140 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbec00cf6f0 con 0x7fbec0160d60 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.526+0000 7fbec6c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbec006e5a0 con 0x7fbec0146280 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58326/0 (socket says 172.21.15.88:58326) 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 -- 172.21.15.88:0/28683171 learned_addr learned my addr 172.21.15.88:0/28683171 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec4e00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbec0160d60 0x7fbec0141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec5800640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0146280 0x7fbec01f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 -- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0146280 msgr2=0x7fbec01f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0146280 0x7fbec01f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 -- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbec0160d60 msgr2=0x7fbec0141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbec0160d60 0x7fbec0141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 -- 172.21.15.88:0/28683171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec013f4a0 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.527+0000 7fbec6200640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.528+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8041020 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.528+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb803d640 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.528+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec013f780 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.528+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8037040 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.528+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec013c730 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.529+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec00693c0 con 0x7fbec0161140 2024-04-26T19:59:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.530+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbeb8044020 con 0x7fbec0161140 2024-04-26T19:59:48.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.531+0000 7fbebd600640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 0x7fbea406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:48.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.531+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbeb80ac2f0 con 0x7fbec0161140 2024-04-26T19:59:48.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.531+0000 7fbec5800640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 0x7fbea406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:48.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.532+0000 7fbec5800640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 0x7fbea406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000f60 tx=0x7fbeb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:48.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.535+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbeb804b030 con 0x7fbec0161140 2024-04-26T19:59:48.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:48.767+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbec0144a80 con 0x7fbec0161140 2024-04-26T19:59:49.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.311+0000 7fbebd600640 1 -- 172.21.15.88:0/28683171 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fbeb8099020 con 0x7fbec0161140 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.314+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 msgr2=0x7fbea406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.314+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 0x7fbea406ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000f60 tx=0x7fbeb0009040 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 msgr2=0x7fbec01f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009870 tx=0x7fbeb80046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 shutdown_connections 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbec0161140 0x7fbec01f8390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbec0160d60 0x7fbec0141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbea406c920 0x7fbea406ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 --2- 172.21.15.88:0/28683171 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbec0146280 0x7fbec01f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 >> 172.21.15.88:0/28683171 conn(0x7fbec00b2900 msgr2=0x7fbec015e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 shutdown_connections 2024-04-26T19:59:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.315+0000 7fbec6c00640 1 -- 172.21.15.88:0/28683171 wait complete. 2024-04-26T19:59:49.338 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T19:59:49.338 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:59:49.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.531+0000 7fac95e00640 1 Processor -- start 2024-04-26T19:59:49.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.531+0000 7fac95e00640 1 -- start start 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac95e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fac90146680 0x7fac900b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac95e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fac900d0e70 0x7fac900d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac95e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac90146a60 con 0x7fac90160930 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac95e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fac9006e970 con 0x7fac90146680 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac95e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fac900d17f0 con 0x7fac900d0e70 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac94a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fac90146680 0x7fac900b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac94a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fac90146680 0x7fac900b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44830/0 (socket says 172.21.15.88:44830) 2024-04-26T19:59:49.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.532+0000 7fac94a00640 1 -- 172.21.15.88:0/911659124 learned_addr learned my addr 172.21.15.88:0/911659124 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:49.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.533+0000 7fac94a00640 1 -- 172.21.15.88:0/911659124 >> v2:172.21.15.132:3300/0 conn(0x7fac900d0e70 msgr2=0x7fac900d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:49.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.533+0000 7fac94a00640 1 --2- 172.21.15.88:0/911659124 >> v2:172.21.15.132:3300/0 conn(0x7fac900d0e70 0x7fac900d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.533+0000 7fac94a00640 1 -- 172.21.15.88:0/911659124 >> v1:172.21.15.132:6789/0 conn(0x7fac90160930 legacy=0x7fac90160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:49.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.533+0000 7fac94a00640 1 -- 172.21.15.88:0/911659124 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9006dd60 con 0x7fac90146680 2024-04-26T19:59:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.534+0000 7fac94a00640 1 --2- 172.21.15.88:0/911659124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90146680 0x7fac900b5990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac90160890 tx=0x7fac8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2948056ac997bded server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac8f400640 1 -- 172.21.15.88:0/911659124 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84037070 con 0x7fac90146680 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac8f400640 1 -- 172.21.15.88:0/911659124 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84032440 con 0x7fac90146680 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac8f400640 1 -- 172.21.15.88:0/911659124 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84032810 con 0x7fac90146680 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 -- 172.21.15.88:0/911659124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90146680 msgr2=0x7fac900b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 --2- 172.21.15.88:0/911659124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90146680 0x7fac900b5990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fac90160890 tx=0x7fac8402a230 comp rx=0 tx=0).stop 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 -- 172.21.15.88:0/911659124 shutdown_connections 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 --2- 172.21.15.88:0/911659124 >> v2:172.21.15.132:3300/0 conn(0x7fac900d0e70 0x7fac900d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 --2- 172.21.15.88:0/911659124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90146680 0x7fac900b5990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 -- 172.21.15.88:0/911659124 >> 172.21.15.88:0/911659124 conn(0x7fac9006c740 msgr2=0x7fac9006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 -- 172.21.15.88:0/911659124 shutdown_connections 2024-04-26T19:59:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.535+0000 7fac95e00640 1 -- 172.21.15.88:0/911659124 wait complete. 2024-04-26T19:59:49.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.537+0000 7fac95e00640 1 Processor -- start 2024-04-26T19:59:49.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.537+0000 7fac95e00640 1 -- start start 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 0x7fac900c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 0x7fac900c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac9006e600 con 0x7fac90160930 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fac9006e320 con 0x7fac900d0e70 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fac900cd6a0 con 0x7fac90146680 2024-04-26T19:59:49.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44840/0 (socket says 172.21.15.88:44840) 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.538+0000 7fac95400640 1 -- 172.21.15.88:0/1263643413 learned_addr learned my addr 172.21.15.88:0/1263643413 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac8fe00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 0x7fac900c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac94a00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 0x7fac900c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 -- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 msgr2=0x7fac900c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 0x7fac900c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 -- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 msgr2=0x7fac900c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 0x7fac900c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 -- 172.21.15.88:0/1263643413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac901e7540 con 0x7fac90160930 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac8fe00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 0x7fac900c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac94a00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 0x7fac900c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95400640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac8800b970 tx=0x7fac8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88019070 con 0x7fac90160930 2024-04-26T19:59:49.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8800cae0 con 0x7fac90160930 2024-04-26T19:59:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac901e7820 con 0x7fac90160930 2024-04-26T19:59:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.539+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88014440 con 0x7fac90160930 2024-04-26T19:59:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.540+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac901e7de0 con 0x7fac90160930 2024-04-26T19:59:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.540+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac900693c0 con 0x7fac90160930 2024-04-26T19:59:49.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.542+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac8801c070 con 0x7fac90160930 2024-04-26T19:59:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.543+0000 7fac8cc00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 0x7fac7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.543+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac8808b870 con 0x7fac90160930 2024-04-26T19:59:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.543+0000 7fac94a00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 0x7fac7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.543+0000 7fac94a00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 0x7fac7006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac84034000 tx=0x7fac8402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:49.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.547+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac88059b70 con 0x7fac90160930 2024-04-26T19:59:49.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.768+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac90130ce0 con 0x7fac90160930 2024-04-26T19:59:49.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.769+0000 7fac8cc00640 1 -- 172.21.15.88:0/1263643413 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7fac88059470 con 0x7fac90160930 2024-04-26T19:59:49.771 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:49.771 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T19:59:49.772 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 msgr2=0x7fac7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 0x7fac7006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac84034000 tx=0x7fac8402e000 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 msgr2=0x7fac90131030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac8800b970 tx=0x7fac8800be30 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 shutdown_connections 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac90160930 0x7fac90131030 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fac7006c920 0x7fac7006ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fac90146680 0x7fac900c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 --2- 172.21.15.88:0/1263643413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac900d0e70 0x7fac900c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 >> 172.21.15.88:0/1263643413 conn(0x7fac9006c740 msgr2=0x7fac900b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:49.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 shutdown_connections 2024-04-26T19:59:49.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.772+0000 7fac95e00640 1 -- 172.21.15.88:0/1263643413 wait complete. 2024-04-26T19:59:49.793 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:49.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.982+0000 7fd1a7200640 1 Processor -- start 2024-04-26T19:59:49.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.983+0000 7fd1a7200640 1 -- start start 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a7200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd1a015dc80 0x7fd1a0167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd1a006c770 0x7fd1a006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a7200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1a0164740 con 0x7fd1a0163b30 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a7200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a00b5ff0 con 0x7fd1a015dc80 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd1a006d110 con 0x7fd1a006c770 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd19fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd1a015dc80 0x7fd1a0167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a5200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd1a0163b30 0x7fd1a0163f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52726/0 (socket says 172.21.15.88:52726) 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.984+0000 7fd1a5200640 1 -- 172.21.15.88:0/3847886610 learned_addr learned my addr 172.21.15.88:0/3847886610 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd197400640 1 --2- 172.21.15.88:0/3847886610 >> v2:172.21.15.132:3300/0 conn(0x7fd1a006c770 0x7fd1a006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd19fe00640 1 -- 172.21.15.88:0/3847886610 >> v2:172.21.15.132:3300/0 conn(0x7fd1a006c770 msgr2=0x7fd1a006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:49.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd19fe00640 1 --2- 172.21.15.88:0/3847886610 >> v2:172.21.15.132:3300/0 conn(0x7fd1a006c770 0x7fd1a006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd19fe00640 1 -- 172.21.15.88:0/3847886610 >> v1:172.21.15.88:6789/0 conn(0x7fd1a0163b30 legacy=0x7fd1a0163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd19fe00640 1 -- 172.21.15.88:0/3847886610 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a01671a0 con 0x7fd1a015dc80 2024-04-26T19:59:49.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.985+0000 7fd19f400640 1 -- 172.21.15.88:0/3847886610 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962263597 0 0) 0x7fd1a0164740 con 0x7fd1a0163b30 2024-04-26T19:59:49.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.986+0000 7fd19fe00640 1 --2- 172.21.15.88:0/3847886610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a0167c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00b3900 tx=0x7fd19002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef78faba21496f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd19f400640 1 -- 172.21.15.88:0/3847886610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190037070 con 0x7fd1a015dc80 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd19f400640 1 -- 172.21.15.88:0/3847886610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190032440 con 0x7fd1a015dc80 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd19f400640 1 -- 172.21.15.88:0/3847886610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19002ae60 con 0x7fd1a015dc80 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 -- 172.21.15.88:0/3847886610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 msgr2=0x7fd1a0167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 --2- 172.21.15.88:0/3847886610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a0167c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00b3900 tx=0x7fd19002a110 comp rx=0 tx=0).stop 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 -- 172.21.15.88:0/3847886610 shutdown_connections 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 --2- 172.21.15.88:0/3847886610 >> v2:172.21.15.132:3300/0 conn(0x7fd1a006c770 0x7fd1a006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 --2- 172.21.15.88:0/3847886610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a0167c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.987+0000 7fd1a7200640 1 -- 172.21.15.88:0/3847886610 >> 172.21.15.88:0/3847886610 conn(0x7fd1a00b2d50 msgr2=0x7fd1a00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.988+0000 7fd1a7200640 1 -- 172.21.15.88:0/3847886610 shutdown_connections 2024-04-26T19:59:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.988+0000 7fd1a7200640 1 -- 172.21.15.88:0/3847886610 wait complete. 2024-04-26T19:59:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.990+0000 7fd1a7200640 1 Processor -- start 2024-04-26T19:59:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.990+0000 7fd1a7200640 1 -- start start 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1a006c770 0x7fd1a009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a0163b30 0x7fd1a009dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0164a20 con 0x7fd1a015dc80 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a0167760 con 0x7fd1a0163b30 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd1a7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd1a00add80 con 0x7fd1a006c770 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd197400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd197400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44854/0 (socket says 172.21.15.88:44854) 2024-04-26T19:59:49.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.991+0000 7fd197400640 1 -- 172.21.15.88:0/1399601639 learned_addr learned my addr 172.21.15.88:0/1399601639 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 -- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1a006c770 msgr2=0x7fd1a009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1a006c770 0x7fd1a009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 -- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a0163b30 msgr2=0x7fd1a009dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a0163b30 0x7fd1a009dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 -- 172.21.15.88:0/1399601639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a009c2a0 con 0x7fd1a015dc80 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd197400640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18800b6c0 con 0x7fd1a015dc80 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd188019070 con 0x7fd1a015dc80 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18800ba80 con 0x7fd1a015dc80 2024-04-26T19:59:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a009c580 con 0x7fd1a015dc80 2024-04-26T19:59:49.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.992+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a009cb40 con 0x7fd1a015dc80 2024-04-26T19:59:49.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.994+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a009cf80 con 0x7fd1a015dc80 2024-04-26T19:59:49.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.994+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd18800bc20 con 0x7fd1a015dc80 2024-04-26T19:59:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.996+0000 7fd19cc00640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.996+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd18808bf50 con 0x7fd1a015dc80 2024-04-26T19:59:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.996+0000 7fd19fe00640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:49.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:49.997+0000 7fd19fe00640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 0x7fd17c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd190034000 tx=0x7fd19002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.000+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd18805a410 con 0x7fd1a015dc80 2024-04-26T19:59:50.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.197+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1a0073fc0 con 0x7fd1a015dc80 2024-04-26T19:59:50.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.200+0000 7fd19cc00640 1 -- 172.21.15.88:0/1399601639 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7fd188059d10 con 0x7fd1a015dc80 2024-04-26T19:59:50.202 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:50.202 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 msgr2=0x7fd17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 0x7fd17c06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd190034000 tx=0x7fd19002c000 comp rx=0 tx=0).stop 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 msgr2=0x7fd1a00b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dcc0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 shutdown_connections 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1a0163b30 0x7fd1a009dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1a015dc80 0x7fd1a00b0730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd17c06c8f0 0x7fd17c06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 --2- 172.21.15.88:0/1399601639 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1a006c770 0x7fd1a009ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 >> 172.21.15.88:0/1399601639 conn(0x7fd1a00b2d50 msgr2=0x7fd1a015c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:50.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 shutdown_connections 2024-04-26T19:59:50.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.203+0000 7fd1a7200640 1 -- 172.21.15.88:0/1399601639 wait complete. 2024-04-26T19:59:50.223 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T19:59:50.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:59:50.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.420+0000 7fe37d800640 1 Processor -- start 2024-04-26T19:59:50.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.420+0000 7fe37d800640 1 -- start start 2024-04-26T19:59:50.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 0x7fe3781638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37806c770 0x7fe37806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe37815f740 0x7fe37815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe378159c80 con 0x7fe37815f740 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe37806d110 con 0x7fe3781598a0 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe37d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe378160110 con 0x7fe37806c770 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe377400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe37815f740 0x7fe37815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe377400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe37815f740 0x7fe37815fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44868/0 (socket says 172.21.15.88:44868) 2024-04-26T19:59:50.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe377400640 1 -- 172.21.15.88:0/3292809969 learned_addr learned my addr 172.21.15.88:0/3292809969 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe376a00640 1 --2- 172.21.15.88:0/3292809969 >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 0x7fe3781638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.422+0000 7fe376000640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37806c770 0x7fe37806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe377400640 1 -- 172.21.15.88:0/3292809969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37806c770 msgr2=0x7fe37806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe377400640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37806c770 0x7fe37806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe377400640 1 -- 172.21.15.88:0/3292809969 >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 msgr2=0x7fe3781638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:50.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe377400640 1 --2- 172.21.15.88:0/3292809969 >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 0x7fe3781638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe377400640 1 -- 172.21.15.88:0/3292809969 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3780b5fb0 con 0x7fe37815f740 2024-04-26T19:59:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.423+0000 7fe376a00640 1 --2- 172.21.15.88:0/3292809969 >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 0x7fe3781638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:50.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.424+0000 7fe377400640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37815f740 0x7fe37815fbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3780b0870 tx=0x7fe37002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11173e05d57552eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.424+0000 7fe375600640 1 -- 172.21.15.88:0/3292809969 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370036070 con 0x7fe37815f740 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.424+0000 7fe375600640 1 -- 172.21.15.88:0/3292809969 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe370031600 con 0x7fe37815f740 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.424+0000 7fe375600640 1 -- 172.21.15.88:0/3292809969 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3700319d0 con 0x7fe37815f740 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 -- 172.21.15.88:0/3292809969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37815f740 msgr2=0x7fe37815fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37815f740 0x7fe37815fbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe3780b0870 tx=0x7fe37002a640 comp rx=0 tx=0).stop 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 -- 172.21.15.88:0/3292809969 shutdown_connections 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37815f740 0x7fe37815fbd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 --2- 172.21.15.88:0/3292809969 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37806c770 0x7fe37806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 --2- 172.21.15.88:0/3292809969 >> v2:172.21.15.132:3300/0 conn(0x7fe3781598a0 0x7fe3781638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 -- 172.21.15.88:0/3292809969 >> 172.21.15.88:0/3292809969 conn(0x7fe3780b2d50 msgr2=0x7fe3780b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:50.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 -- 172.21.15.88:0/3292809969 shutdown_connections 2024-04-26T19:59:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.425+0000 7fe37d800640 1 -- 172.21.15.88:0/3292809969 wait complete. 2024-04-26T19:59:50.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.427+0000 7fe37d800640 1 Processor -- start 2024-04-26T19:59:50.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.427+0000 7fe37d800640 1 -- start start 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.428+0000 7fe37d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.428+0000 7fe37d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 0x7fe37809dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.428+0000 7fe37d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 0x7fe3781f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe37d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781630b0 con 0x7fe37806c770 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe37d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe378162dd0 con 0x7fe3781598a0 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe37d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe378163390 con 0x7fe37815f740 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44878/0 (socket says 172.21.15.88:44878) 2024-04-26T19:59:50.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 -- 172.21.15.88:0/2714473276 learned_addr learned my addr 172.21.15.88:0/2714473276 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe377400640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 0x7fe3781f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 -- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 msgr2=0x7fe3781f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 0x7fe3781f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376000640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 0x7fe37809dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 -- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 msgr2=0x7fe37809dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 0x7fe37809dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376a00640 1 -- 172.21.15.88:0/2714473276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37809be30 con 0x7fe37806c770 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe377400640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 0x7fe3781f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.429+0000 7fe376000640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 0x7fe37809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe376a00640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c7f0 tx=0x7fe36800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368015070 con 0x7fe37806c770 2024-04-26T19:59:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe368007a30 con 0x7fe37806c770 2024-04-26T19:59:50.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368007e00 con 0x7fe37806c770 2024-04-26T19:59:50.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37809c110 con 0x7fe37806c770 2024-04-26T19:59:50.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.430+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37809c6d0 con 0x7fe37806c770 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.431+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe378069540 con 0x7fe37806c770 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.432+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe368018070 con 0x7fe37806c770 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.433+0000 7fe36ea00640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 0x7fe35806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.433+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe36808bbb0 con 0x7fe37806c770 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.433+0000 7fe376000640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 0x7fe35806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.434+0000 7fe376000640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 0x7fe35806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe378147260 tx=0x7fe364008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.438+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe368059f40 con 0x7fe37806c770 2024-04-26T19:59:50.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.649+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe378159060 con 0x7fe35806c8f0 2024-04-26T19:59:50.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.658+0000 7fe36ea00640 1 -- 172.21.15.88:0/2714473276 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240068 (secure 0 0 0) 0x7fe378159060 con 0x7fe35806c8f0 2024-04-26T19:59:50.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:50.665 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T19:59:50.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 msgr2=0x7fe35806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 0x7fe35806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe378147260 tx=0x7fe364008040 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 msgr2=0x7fe3781f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe36800c7f0 tx=0x7fe36800cbf0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 shutdown_connections 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe35806c8f0 0x7fe35806edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe37815f740 0x7fe3781f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3781598a0 0x7fe37809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 --2- 172.21.15.88:0/2714473276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe37806c770 0x7fe3781f6430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 >> 172.21.15.88:0/2714473276 conn(0x7fe3780b2d50 msgr2=0x7fe3780b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 shutdown_connections 2024-04-26T19:59:50.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.664+0000 7fe37d800640 1 -- 172.21.15.88:0/2714473276 wait complete. 2024-04-26T19:59:50.686 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T19:59:48.941330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218632,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756088,"statfs":{"total":773094113280,"available":772870234112,"internally_reserved":0,"allocated":2531328,"data_stored":1593408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.702406"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.244978+0000","last_change":"2024-04-26T19:59:44.244978+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.244978+0000","last_undegraded":"2024-04-26T19:59:44.244978+0000","last_fullsized":"2024-04-26T19:59:44.244978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254492+0000","last_change":"2024-04-26T19:59:44.254492+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254492+0000","last_undegraded":"2024-04-26T19:59:44.254492+0000","last_fullsized":"2024-04-26T19:59:44.254492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.245000+0000","last_change":"2024-04-26T19:59:44.245000+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.245000+0000","last_undegraded":"2024-04-26T19:59:44.245000+0000","last_fullsized":"2024-04-26T19:59:44.245000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.264475+0000","last_change":"2024-04-26T19:59:44.264475+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.264475+0000","last_undegraded":"2024-04-26T19:59:44.264475+0000","last_fullsized":"2024-04-26T19:59:44.264475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.256138+0000","last_change":"2024-04-26T19:59:44.256138+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.256138+0000","last_undegraded":"2024-04-26T19:59:44.256138+0000","last_fullsized":"2024-04-26T19:59:44.256138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.263148+0000","last_change":"2024-04-26T19:59:44.263148+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.263148+0000","last_undegraded":"2024-04-26T19:59:44.263148+0000","last_fullsized":"2024-04-26T19:59:44.263148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.255781+0000","last_change":"2024-04-26T19:59:44.255781+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.255781+0000","last_undegraded":"2024-04-26T19:59:44.255781+0000","last_fullsized":"2024-04-26T19:59:44.255781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.256131+0000","last_change":"2024-04-26T19:59:44.256131+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.256131+0000","last_undegraded":"2024-04-26T19:59:44.256131+0000","last_fullsized":"2024-04-26T19:59:44.256131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.248660+0000","last_change":"2024-04-26T19:59:44.248660+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.248660+0000","last_undegraded":"2024-04-26T19:59:44.248660+0000","last_fullsized":"2024-04-26T19:59:44.248660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.264291+0000","last_change":"2024-04-26T19:59:44.264291+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.264291+0000","last_undegraded":"2024-04-26T19:59:44.264291+0000","last_fullsized":"2024-04-26T19:59:44.264291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254509+0000","last_change":"2024-04-26T19:59:44.254509+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254509+0000","last_undegraded":"2024-04-26T19:59:44.254509+0000","last_fullsized":"2024-04-26T19:59:44.254509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.267205+0000","last_change":"2024-04-26T19:59:44.267205+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.267205+0000","last_undegraded":"2024-04-26T19:59:44.267205+0000","last_fullsized":"2024-04-26T19:59:44.267205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.246772+0000","last_change":"2024-04-26T19:59:44.246772+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.246772+0000","last_undegraded":"2024-04-26T19:59:44.246772+0000","last_fullsized":"2024-04-26T19:59:44.246772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.267912+0000","last_change":"2024-04-26T19:59:44.267912+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.267912+0000","last_undegraded":"2024-04-26T19:59:44.267912+0000","last_fullsized":"2024-04-26T19:59:44.267912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254425+0000","last_change":"2024-04-26T19:59:44.254425+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254425+0000","last_undegraded":"2024-04-26T19:59:44.254425+0000","last_fullsized":"2024-04-26T19:59:44.254425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.266666+0000","last_change":"2024-04-26T19:59:44.266666+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.266666+0000","last_undegraded":"2024-04-26T19:59:44.266666+0000","last_fullsized":"2024-04-26T19:59:44.266666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T19:59:44.253928+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T19:59:44.253928+0000","last_peered":"2024-04-26T19:59:44.253928+0000","last_clean":"2024-04-26T19:59:44.253928+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T19:59:44.253928+0000","last_undegraded":"2024-04-26T19:59:44.253928+0000","last_fullsized":"2024-04-26T19:59:44.253928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.262677+0000","last_change":"2024-04-26T19:59:44.262677+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.262677+0000","last_undegraded":"2024-04-26T19:59:44.262677+0000","last_fullsized":"2024-04-26T19:59:44.262677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.247876+0000","last_change":"2024-04-26T19:59:44.247876+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.247876+0000","last_undegraded":"2024-04-26T19:59:44.247876+0000","last_fullsized":"2024-04-26T19:59:44.247876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.267646+0000","last_change":"2024-04-26T19:59:44.267646+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.267646+0000","last_undegraded":"2024-04-26T19:59:44.267646+0000","last_fullsized":"2024-04-26T19:59:44.267646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.262597+0000","last_change":"2024-04-26T19:59:44.262597+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.262597+0000","last_undegraded":"2024-04-26T19:59:44.262597+0000","last_fullsized":"2024-04-26T19:59:44.262597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254333+0000","last_change":"2024-04-26T19:59:44.254333+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254333+0000","last_undegraded":"2024-04-26T19:59:44.254333+0000","last_fullsized":"2024-04-26T19:59:44.254333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.248195+0000","last_change":"2024-04-26T19:59:44.248195+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.248195+0000","last_undegraded":"2024-04-26T19:59:44.248195+0000","last_fullsized":"2024-04-26T19:59:44.248195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.255402+0000","last_change":"2024-04-26T19:59:44.255402+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.255402+0000","last_undegraded":"2024-04-26T19:59:44.255402+0000","last_fullsized":"2024-04-26T19:59:44.255402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254453+0000","last_change":"2024-04-26T19:59:44.254453+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254453+0000","last_undegraded":"2024-04-26T19:59:44.254453+0000","last_fullsized":"2024-04-26T19:59:44.254453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.250181+0000","last_change":"2024-04-26T19:59:44.250181+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.250181+0000","last_undegraded":"2024-04-26T19:59:44.250181+0000","last_fullsized":"2024-04-26T19:59:44.250181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.254125+0000","last_change":"2024-04-26T19:59:44.254125+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.254125+0000","last_undegraded":"2024-04-26T19:59:44.254125+0000","last_fullsized":"2024-04-26T19:59:44.254125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.253725+0000","last_change":"2024-04-26T19:59:44.253725+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.253725+0000","last_undegraded":"2024-04-26T19:59:44.253725+0000","last_fullsized":"2024-04-26T19:59:44.253725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.248190+0000","last_change":"2024-04-26T19:59:44.248190+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.248190+0000","last_undegraded":"2024-04-26T19:59:44.248190+0000","last_fullsized":"2024-04-26T19:59:44.248190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.249871+0000","last_change":"2024-04-26T19:59:44.249871+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.249871+0000","last_undegraded":"2024-04-26T19:59:44.249871+0000","last_fullsized":"2024-04-26T19:59:44.249871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.251596+0000","last_change":"2024-04-26T19:59:44.251596+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.251596+0000","last_undegraded":"2024-04-26T19:59:44.251596+0000","last_fullsized":"2024-04-26T19:59:44.251596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.245000+0000","last_change":"2024-04-26T19:59:44.245000+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.245000+0000","last_undegraded":"2024-04-26T19:59:44.245000+0000","last_fullsized":"2024-04-26T19:59:44.245000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:59:45.238672+0000","last_change":"2024-04-26T19:59:45.238672+0000","last_active":"2024-04-26T19:59:45.238672+0000","last_peered":"2024-04-26T19:59:45.238672+0000","last_clean":"2024-04-26T19:59:45.238672+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:45.238672+0000","last_undegraded":"2024-04-26T19:59:45.238672+0000","last_fullsized":"2024-04-26T19:59:45.238672+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2024-04-26T19:59:44.248840+0000","last_change":"2024-04-26T19:59:44.248840+0000","last_active":"2024-04-26T19:59:44.223039+0000","last_peered":"2024-04-26T19:59:44.223039+0000","last_clean":"2024-04-26T19:59:44.223039+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:59:44.248840+0000","last_undegraded":"2024-04-26T19:59:44.248840+0000","last_fullsized":"2024-04-26T19:59:44.248840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:59:50.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:59:50.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.928+0000 7f998ca00640 1 Processor -- start 2024-04-26T19:59:50.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.929+0000 7f998ca00640 1 -- start start 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f998ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f9988073b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f998ca00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880d0ab0 con 0x7f9988070640 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f998ca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99880d0d90 con 0x7f99881468f0 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f998ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f99880d1070 con 0x7f998806ca60 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f9985e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9988070640 0x7f998806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52734/0 (socket says 172.21.15.88:52734) 2024-04-26T19:59:50.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.930+0000 7f9985e00640 1 -- 172.21.15.88:0/1225889289 learned_addr learned my addr 172.21.15.88:0/1225889289 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9987200640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f9988073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695379634 0 0) 0x7f99880d0ab0 con 0x7f9988070640 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f996c003520 con 0x7f9988070640 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900157175 0 0) 0x7f99880d0d90 con 0x7f99881468f0 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99880d0ab0 con 0x7f99881468f0 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3514422656 0 0) 0x7f996c003520 con 0x7f9988070640 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 msgr2=0x7f9988073b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f9988073b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 >> v1:172.21.15.132:6789/0 conn(0x7f99881468f0 legacy=0x7f99880d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880d1350 con 0x7f9988070640 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f9974001690 con 0x7f9988070640 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 >> v1:172.21.15.88:6789/0 conn(0x7f9988070640 legacy=0x7f998806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.931+0000 7f9985400640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f996c003b70 0x7f996c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9985400640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c004800 0x7f996c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9985400640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f996c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f99880d1070 con 0x7f996c003b70 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f996c00baa0 con 0x7f996c004800 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f996c00bca0 con 0x7f998806ca60 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f996c003b70 0x7f996c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9987200640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f996c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 -- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 msgr2=0x7f996c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f996c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 -- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c004800 msgr2=0x7f996c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c004800 0x7f996c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9986800640 1 -- 172.21.15.88:0/1225889289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c003520 con 0x7f996c003b70 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.932+0000 7f9987200640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998806ca60 0x7f996c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.933+0000 7f9986800640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f996c003b70 0x7f996c003f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9978001fa0 tx=0x7f996c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a373c922ff67c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.933+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978001bf0 con 0x7f996c003b70 2024-04-26T19:59:50.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.933+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9978003b70 con 0x7f996c003b70 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.933+0000 7f9985400640 1 -- 172.21.15.88:0/1225889289 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99780050d0 con 0x7f996c003b70 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 -- 172.21.15.88:0/1225889289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f996c003b70 msgr2=0x7f996c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 --2- 172.21.15.88:0/1225889289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f996c003b70 0x7f996c003f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9978001fa0 tx=0x7f996c005830 comp rx=0 tx=0).stop 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f9987200640 1 -- 172.21.15.88:0/1225889289 reap_dead start 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 -- 172.21.15.88:0/1225889289 shutdown_connections 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 -- 172.21.15.88:0/1225889289 >> 172.21.15.88:0/1225889289 conn(0x7f99880b38a0 msgr2=0x7f99880b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 -- 172.21.15.88:0/1225889289 shutdown_connections 2024-04-26T19:59:50.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.934+0000 7f998ca00640 1 -- 172.21.15.88:0/1225889289 wait complete. 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.936+0000 7f998ca00640 1 Processor -- start 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.936+0000 7f998ca00640 1 -- start start 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c003b70 0x7f99881f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998815c1c0 0x7f99881f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9988073f30 con 0x7f9988133b60 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9988073db0 con 0x7f996c003b70 2024-04-26T19:59:50.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f998ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9988165a10 con 0x7f998815c1c0 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f9985e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f9985e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44894/0 (socket says 172.21.15.88:44894) 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f9985e00640 1 -- 172.21.15.88:0/3557635816 learned_addr learned my addr 172.21.15.88:0/3557635816 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.937+0000 7f9985e00640 1 -- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998815c1c0 msgr2=0x7f99881f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f9985e00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998815c1c0 0x7f99881f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f9985e00640 1 -- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c003b70 msgr2=0x7f99881f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f9985e00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c003b70 0x7f99881f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f9985e00640 1 -- 172.21.15.88:0/3557635816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99881f8680 con 0x7f9988133b60 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f9985e00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f997400c760 tx=0x7f997400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974018070 con 0x7f9988133b60 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99740074e0 con 0x7f9988133b60 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974004a20 con 0x7f9988133b60 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99881f9580 con 0x7f9988133b60 2024-04-26T19:59:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.938+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9988133360 con 0x7f9988133b60 2024-04-26T19:59:50.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.939+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9988068920 con 0x7f9988133b60 2024-04-26T19:59:50.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.940+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9974004bc0 con 0x7f9988133b60 2024-04-26T19:59:50.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.941+0000 7f997ea00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:50.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.941+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f997408af80 con 0x7f9988133b60 2024-04-26T19:59:50.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.942+0000 7f9986800640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:50.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.942+0000 7f9986800640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9978032000 tx=0x7f9978009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:50.946+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9988068920 con 0x7f9988133b60 2024-04-26T19:59:51.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.144+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9988132e70 con 0x7f996806c8f0 2024-04-26T19:59:51.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.163+0000 7f997ea00640 1 -- 172.21.15.88:0/3557635816 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f9988132e70 con 0x7f996806c8f0 2024-04-26T19:59:51.168 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:51.170 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 msgr2=0x7f996806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 0x7f996806edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9978032000 tx=0x7f9978009030 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 msgr2=0x7f998815bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f997400c760 tx=0x7f997400cc20 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 shutdown_connections 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f996806c8f0 0x7f996806edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f998815c1c0 0x7f99881f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9988133b60 0x7f998815bc80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 --2- 172.21.15.88:0/3557635816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f996c003b70 0x7f99881f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 >> 172.21.15.88:0/3557635816 conn(0x7f99880b38a0 msgr2=0x7f9988164d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.168+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 shutdown_connections 2024-04-26T19:59:51.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.169+0000 7f998ca00640 1 -- 172.21.15.88:0/3557635816 wait complete. 2024-04-26T19:59:51.190 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T19:59:50.941951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001974"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465394+0000","last_change":"2024-04-26T19:59:46.299176+0000","last_active":"2024-04-26T19:59:47.465394+0000","last_peered":"2024-04-26T19:59:47.465394+0000","last_clean":"2024-04-26T19:59:47.465394+0000","last_became_active":"2024-04-26T19:59:46.298473+0000","last_became_peered":"2024-04-26T19:59:46.298473+0000","last_unstale":"2024-04-26T19:59:47.465394+0000","last_undegraded":"2024-04-26T19:59:47.465394+0000","last_fullsized":"2024-04-26T19:59:47.465394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294427+0000","last_change":"2024-04-26T19:59:46.279518+0000","last_active":"2024-04-26T19:59:47.294427+0000","last_peered":"2024-04-26T19:59:47.294427+0000","last_clean":"2024-04-26T19:59:47.294427+0000","last_became_active":"2024-04-26T19:59:46.279235+0000","last_became_peered":"2024-04-26T19:59:46.279235+0000","last_unstale":"2024-04-26T19:59:47.294427+0000","last_undegraded":"2024-04-26T19:59:47.294427+0000","last_fullsized":"2024-04-26T19:59:47.294427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465545+0000","last_change":"2024-04-26T19:59:46.293998+0000","last_active":"2024-04-26T19:59:47.465545+0000","last_peered":"2024-04-26T19:59:47.465545+0000","last_clean":"2024-04-26T19:59:47.465545+0000","last_became_active":"2024-04-26T19:59:46.293698+0000","last_became_peered":"2024-04-26T19:59:46.293698+0000","last_unstale":"2024-04-26T19:59:47.465545+0000","last_undegraded":"2024-04-26T19:59:47.465545+0000","last_fullsized":"2024-04-26T19:59:47.465545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465774+0000","last_change":"2024-04-26T19:59:46.295413+0000","last_active":"2024-04-26T19:59:47.465774+0000","last_peered":"2024-04-26T19:59:47.465774+0000","last_clean":"2024-04-26T19:59:47.465774+0000","last_became_active":"2024-04-26T19:59:46.294973+0000","last_became_peered":"2024-04-26T19:59:46.294973+0000","last_unstale":"2024-04-26T19:59:47.465774+0000","last_undegraded":"2024-04-26T19:59:47.465774+0000","last_fullsized":"2024-04-26T19:59:47.465774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469290+0000","last_change":"2024-04-26T19:59:46.299569+0000","last_active":"2024-04-26T19:59:47.469290+0000","last_peered":"2024-04-26T19:59:47.469290+0000","last_clean":"2024-04-26T19:59:47.469290+0000","last_became_active":"2024-04-26T19:59:46.299229+0000","last_became_peered":"2024-04-26T19:59:46.299229+0000","last_unstale":"2024-04-26T19:59:47.469290+0000","last_undegraded":"2024-04-26T19:59:47.469290+0000","last_fullsized":"2024-04-26T19:59:47.469290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464401+0000","last_change":"2024-04-26T19:59:46.287705+0000","last_active":"2024-04-26T19:59:47.464401+0000","last_peered":"2024-04-26T19:59:47.464401+0000","last_clean":"2024-04-26T19:59:47.464401+0000","last_became_active":"2024-04-26T19:59:46.287398+0000","last_became_peered":"2024-04-26T19:59:46.287398+0000","last_unstale":"2024-04-26T19:59:47.464401+0000","last_undegraded":"2024-04-26T19:59:47.464401+0000","last_fullsized":"2024-04-26T19:59:47.464401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466440+0000","last_change":"2024-04-26T19:59:46.304005+0000","last_active":"2024-04-26T19:59:47.466440+0000","last_peered":"2024-04-26T19:59:47.466440+0000","last_clean":"2024-04-26T19:59:47.466440+0000","last_became_active":"2024-04-26T19:59:46.303696+0000","last_became_peered":"2024-04-26T19:59:46.303696+0000","last_unstale":"2024-04-26T19:59:47.466440+0000","last_undegraded":"2024-04-26T19:59:47.466440+0000","last_fullsized":"2024-04-26T19:59:47.466440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465106+0000","last_change":"2024-04-26T19:59:46.294993+0000","last_active":"2024-04-26T19:59:47.465106+0000","last_peered":"2024-04-26T19:59:47.465106+0000","last_clean":"2024-04-26T19:59:47.465106+0000","last_became_active":"2024-04-26T19:59:46.294405+0000","last_became_peered":"2024-04-26T19:59:46.294405+0000","last_unstale":"2024-04-26T19:59:47.465106+0000","last_undegraded":"2024-04-26T19:59:47.465106+0000","last_fullsized":"2024-04-26T19:59:47.465106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465470+0000","last_change":"2024-04-26T19:59:46.311322+0000","last_active":"2024-04-26T19:59:47.465470+0000","last_peered":"2024-04-26T19:59:47.465470+0000","last_clean":"2024-04-26T19:59:47.465470+0000","last_became_active":"2024-04-26T19:59:46.308499+0000","last_became_peered":"2024-04-26T19:59:46.308499+0000","last_unstale":"2024-04-26T19:59:47.465470+0000","last_undegraded":"2024-04-26T19:59:47.465470+0000","last_fullsized":"2024-04-26T19:59:47.465470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469431+0000","last_change":"2024-04-26T19:59:46.292325+0000","last_active":"2024-04-26T19:59:47.469431+0000","last_peered":"2024-04-26T19:59:47.469431+0000","last_clean":"2024-04-26T19:59:47.469431+0000","last_became_active":"2024-04-26T19:59:46.288719+0000","last_became_peered":"2024-04-26T19:59:46.288719+0000","last_unstale":"2024-04-26T19:59:47.469431+0000","last_undegraded":"2024-04-26T19:59:47.469431+0000","last_fullsized":"2024-04-26T19:59:47.469431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465975+0000","last_change":"2024-04-26T19:59:46.290820+0000","last_active":"2024-04-26T19:59:47.465975+0000","last_peered":"2024-04-26T19:59:47.465975+0000","last_clean":"2024-04-26T19:59:47.465975+0000","last_became_active":"2024-04-26T19:59:46.289935+0000","last_became_peered":"2024-04-26T19:59:46.289935+0000","last_unstale":"2024-04-26T19:59:47.465975+0000","last_undegraded":"2024-04-26T19:59:47.465975+0000","last_fullsized":"2024-04-26T19:59:47.465975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298228+0000","last_change":"2024-04-26T19:59:46.307273+0000","last_active":"2024-04-26T19:59:47.298228+0000","last_peered":"2024-04-26T19:59:47.298228+0000","last_clean":"2024-04-26T19:59:47.298228+0000","last_became_active":"2024-04-26T19:59:46.306426+0000","last_became_peered":"2024-04-26T19:59:46.306426+0000","last_unstale":"2024-04-26T19:59:47.298228+0000","last_undegraded":"2024-04-26T19:59:47.298228+0000","last_fullsized":"2024-04-26T19:59:47.298228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:35.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463934+0000","last_change":"2024-04-26T19:59:46.304263+0000","last_active":"2024-04-26T19:59:47.463934+0000","last_peered":"2024-04-26T19:59:47.463934+0000","last_clean":"2024-04-26T19:59:47.463934+0000","last_became_active":"2024-04-26T19:59:46.303977+0000","last_became_peered":"2024-04-26T19:59:46.303977+0000","last_unstale":"2024-04-26T19:59:47.463934+0000","last_undegraded":"2024-04-26T19:59:47.463934+0000","last_fullsized":"2024-04-26T19:59:47.463934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464499+0000","last_change":"2024-04-26T19:59:46.280420+0000","last_active":"2024-04-26T19:59:47.464499+0000","last_peered":"2024-04-26T19:59:47.464499+0000","last_clean":"2024-04-26T19:59:47.464499+0000","last_became_active":"2024-04-26T19:59:46.279398+0000","last_became_peered":"2024-04-26T19:59:46.279398+0000","last_unstale":"2024-04-26T19:59:47.464499+0000","last_undegraded":"2024-04-26T19:59:47.464499+0000","last_fullsized":"2024-04-26T19:59:47.464499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293703+0000","last_change":"2024-04-26T19:59:46.310086+0000","last_active":"2024-04-26T19:59:47.293703+0000","last_peered":"2024-04-26T19:59:47.293703+0000","last_clean":"2024-04-26T19:59:47.293703+0000","last_became_active":"2024-04-26T19:59:46.309760+0000","last_became_peered":"2024-04-26T19:59:46.309760+0000","last_unstale":"2024-04-26T19:59:47.293703+0000","last_undegraded":"2024-04-26T19:59:47.293703+0000","last_fullsized":"2024-04-26T19:59:47.293703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464282+0000","last_change":"2024-04-26T19:59:46.290194+0000","last_active":"2024-04-26T19:59:47.464282+0000","last_peered":"2024-04-26T19:59:47.464282+0000","last_clean":"2024-04-26T19:59:47.464282+0000","last_became_active":"2024-04-26T19:59:46.289473+0000","last_became_peered":"2024-04-26T19:59:46.289473+0000","last_unstale":"2024-04-26T19:59:47.464282+0000","last_undegraded":"2024-04-26T19:59:47.464282+0000","last_fullsized":"2024-04-26T19:59:47.464282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468794+0000","last_change":"2024-04-26T19:59:46.312802+0000","last_active":"2024-04-26T19:59:47.468794+0000","last_peered":"2024-04-26T19:59:47.468794+0000","last_clean":"2024-04-26T19:59:47.468794+0000","last_became_active":"2024-04-26T19:59:46.312497+0000","last_became_peered":"2024-04-26T19:59:46.312497+0000","last_unstale":"2024-04-26T19:59:47.468794+0000","last_undegraded":"2024-04-26T19:59:47.468794+0000","last_fullsized":"2024-04-26T19:59:47.468794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297946+0000","last_change":"2024-04-26T19:59:46.290353+0000","last_active":"2024-04-26T19:59:47.297946+0000","last_peered":"2024-04-26T19:59:47.297946+0000","last_clean":"2024-04-26T19:59:47.297946+0000","last_became_active":"2024-04-26T19:59:46.289992+0000","last_became_peered":"2024-04-26T19:59:46.289992+0000","last_unstale":"2024-04-26T19:59:47.297946+0000","last_undegraded":"2024-04-26T19:59:47.297946+0000","last_fullsized":"2024-04-26T19:59:47.297946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468211+0000","last_change":"2024-04-26T19:59:46.302429+0000","last_active":"2024-04-26T19:59:47.468211+0000","last_peered":"2024-04-26T19:59:47.468211+0000","last_clean":"2024-04-26T19:59:47.468211+0000","last_became_active":"2024-04-26T19:59:46.302123+0000","last_became_peered":"2024-04-26T19:59:46.302123+0000","last_unstale":"2024-04-26T19:59:47.468211+0000","last_undegraded":"2024-04-26T19:59:47.468211+0000","last_fullsized":"2024-04-26T19:59:47.468211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468516+0000","last_change":"2024-04-26T19:59:46.300571+0000","last_active":"2024-04-26T19:59:47.468516+0000","last_peered":"2024-04-26T19:59:47.468516+0000","last_clean":"2024-04-26T19:59:47.468516+0000","last_became_active":"2024-04-26T19:59:46.300247+0000","last_became_peered":"2024-04-26T19:59:46.300247+0000","last_unstale":"2024-04-26T19:59:47.468516+0000","last_undegraded":"2024-04-26T19:59:47.468516+0000","last_fullsized":"2024-04-26T19:59:47.468516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464971+0000","last_change":"2024-04-26T19:59:46.300279+0000","last_active":"2024-04-26T19:59:47.464971+0000","last_peered":"2024-04-26T19:59:47.464971+0000","last_clean":"2024-04-26T19:59:47.464971+0000","last_became_active":"2024-04-26T19:59:46.300053+0000","last_became_peered":"2024-04-26T19:59:46.300053+0000","last_unstale":"2024-04-26T19:59:47.464971+0000","last_undegraded":"2024-04-26T19:59:47.464971+0000","last_fullsized":"2024-04-26T19:59:47.464971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466037+0000","last_change":"2024-04-26T19:59:46.305198+0000","last_active":"2024-04-26T19:59:47.466037+0000","last_peered":"2024-04-26T19:59:47.466037+0000","last_clean":"2024-04-26T19:59:47.466037+0000","last_became_active":"2024-04-26T19:59:46.303321+0000","last_became_peered":"2024-04-26T19:59:46.303321+0000","last_unstale":"2024-04-26T19:59:47.466037+0000","last_undegraded":"2024-04-26T19:59:47.466037+0000","last_fullsized":"2024-04-26T19:59:47.466037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463271+0000","last_change":"2024-04-26T19:59:46.313226+0000","last_active":"2024-04-26T19:59:47.463271+0000","last_peered":"2024-04-26T19:59:47.463271+0000","last_clean":"2024-04-26T19:59:47.463271+0000","last_became_active":"2024-04-26T19:59:46.312933+0000","last_became_peered":"2024-04-26T19:59:46.312933+0000","last_unstale":"2024-04-26T19:59:47.463271+0000","last_undegraded":"2024-04-26T19:59:47.463271+0000","last_fullsized":"2024-04-26T19:59:47.463271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464413+0000","last_change":"2024-04-26T19:59:46.296413+0000","last_active":"2024-04-26T19:59:47.464413+0000","last_peered":"2024-04-26T19:59:47.464413+0000","last_clean":"2024-04-26T19:59:47.464413+0000","last_became_active":"2024-04-26T19:59:46.295760+0000","last_became_peered":"2024-04-26T19:59:46.295760+0000","last_unstale":"2024-04-26T19:59:47.464413+0000","last_undegraded":"2024-04-26T19:59:47.464413+0000","last_fullsized":"2024-04-26T19:59:47.464413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465037+0000","last_change":"2024-04-26T19:59:46.314022+0000","last_active":"2024-04-26T19:59:47.465037+0000","last_peered":"2024-04-26T19:59:47.465037+0000","last_clean":"2024-04-26T19:59:47.465037+0000","last_became_active":"2024-04-26T19:59:46.313768+0000","last_became_peered":"2024-04-26T19:59:46.313768+0000","last_unstale":"2024-04-26T19:59:47.465037+0000","last_undegraded":"2024-04-26T19:59:47.465037+0000","last_fullsized":"2024-04-26T19:59:47.465037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469629+0000","last_change":"2024-04-26T19:59:46.299335+0000","last_active":"2024-04-26T19:59:47.469629+0000","last_peered":"2024-04-26T19:59:47.469629+0000","last_clean":"2024-04-26T19:59:47.469629+0000","last_became_active":"2024-04-26T19:59:46.299048+0000","last_became_peered":"2024-04-26T19:59:46.299048+0000","last_unstale":"2024-04-26T19:59:47.469629+0000","last_undegraded":"2024-04-26T19:59:47.469629+0000","last_fullsized":"2024-04-26T19:59:47.469629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294167+0000","last_change":"2024-04-26T19:59:46.297650+0000","last_active":"2024-04-26T19:59:47.294167+0000","last_peered":"2024-04-26T19:59:47.294167+0000","last_clean":"2024-04-26T19:59:47.294167+0000","last_became_active":"2024-04-26T19:59:46.297368+0000","last_became_peered":"2024-04-26T19:59:46.297368+0000","last_unstale":"2024-04-26T19:59:47.294167+0000","last_undegraded":"2024-04-26T19:59:47.294167+0000","last_fullsized":"2024-04-26T19:59:47.294167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464415+0000","last_change":"2024-04-26T19:59:46.313265+0000","last_active":"2024-04-26T19:59:47.464415+0000","last_peered":"2024-04-26T19:59:47.464415+0000","last_clean":"2024-04-26T19:59:47.464415+0000","last_became_active":"2024-04-26T19:59:46.313035+0000","last_became_peered":"2024-04-26T19:59:46.313035+0000","last_unstale":"2024-04-26T19:59:47.464415+0000","last_undegraded":"2024-04-26T19:59:47.464415+0000","last_fullsized":"2024-04-26T19:59:47.464415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465117+0000","last_change":"2024-04-26T19:59:46.294814+0000","last_active":"2024-04-26T19:59:47.465117+0000","last_peered":"2024-04-26T19:59:47.465117+0000","last_clean":"2024-04-26T19:59:47.465117+0000","last_became_active":"2024-04-26T19:59:46.294277+0000","last_became_peered":"2024-04-26T19:59:46.294277+0000","last_unstale":"2024-04-26T19:59:47.465117+0000","last_undegraded":"2024-04-26T19:59:47.465117+0000","last_fullsized":"2024-04-26T19:59:47.465117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466341+0000","last_change":"2024-04-26T19:59:46.297299+0000","last_active":"2024-04-26T19:59:47.466341+0000","last_peered":"2024-04-26T19:59:47.466341+0000","last_clean":"2024-04-26T19:59:47.466341+0000","last_became_active":"2024-04-26T19:59:46.296375+0000","last_became_peered":"2024-04-26T19:59:46.296375+0000","last_unstale":"2024-04-26T19:59:47.466341+0000","last_undegraded":"2024-04-26T19:59:47.466341+0000","last_fullsized":"2024-04-26T19:59:47.466341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297656+0000","last_change":"2024-04-26T19:59:46.292257+0000","last_active":"2024-04-26T19:59:47.297656+0000","last_peered":"2024-04-26T19:59:47.297656+0000","last_clean":"2024-04-26T19:59:47.297656+0000","last_became_active":"2024-04-26T19:59:46.291980+0000","last_became_peered":"2024-04-26T19:59:46.291980+0000","last_unstale":"2024-04-26T19:59:47.297656+0000","last_undegraded":"2024-04-26T19:59:47.297656+0000","last_fullsized":"2024-04-26T19:59:47.297656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464925+0000","last_change":"2024-04-26T19:59:46.283803+0000","last_active":"2024-04-26T19:59:47.464925+0000","last_peered":"2024-04-26T19:59:47.464925+0000","last_clean":"2024-04-26T19:59:47.464925+0000","last_became_active":"2024-04-26T19:59:46.283455+0000","last_became_peered":"2024-04-26T19:59:46.283455+0000","last_unstale":"2024-04-26T19:59:47.464925+0000","last_undegraded":"2024-04-26T19:59:47.464925+0000","last_fullsized":"2024-04-26T19:59:47.464925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297641+0000","last_change":"2024-04-26T19:59:46.290324+0000","last_active":"2024-04-26T19:59:47.297641+0000","last_peered":"2024-04-26T19:59:47.297641+0000","last_clean":"2024-04-26T19:59:47.297641+0000","last_became_active":"2024-04-26T19:59:46.289926+0000","last_became_peered":"2024-04-26T19:59:46.289926+0000","last_unstale":"2024-04-26T19:59:47.297641+0000","last_undegraded":"2024-04-26T19:59:47.297641+0000","last_fullsized":"2024-04-26T19:59:47.297641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:56.862425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464081+0000","last_change":"2024-04-26T19:59:45.291916+0000","last_active":"2024-04-26T19:59:47.464081+0000","last_peered":"2024-04-26T19:59:47.464081+0000","last_clean":"2024-04-26T19:59:47.464081+0000","last_became_active":"2024-04-26T19:59:45.291242+0000","last_became_peered":"2024-04-26T19:59:45.291242+0000","last_unstale":"2024-04-26T19:59:47.464081+0000","last_undegraded":"2024-04-26T19:59:47.464081+0000","last_fullsized":"2024-04-26T19:59:47.464081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465265+0000","last_change":"2024-04-26T19:59:46.295081+0000","last_active":"2024-04-26T19:59:47.465265+0000","last_peered":"2024-04-26T19:59:47.465265+0000","last_clean":"2024-04-26T19:59:47.465265+0000","last_became_active":"2024-04-26T19:59:46.294800+0000","last_became_peered":"2024-04-26T19:59:46.294800+0000","last_unstale":"2024-04-26T19:59:47.465265+0000","last_undegraded":"2024-04-26T19:59:47.465265+0000","last_fullsized":"2024-04-26T19:59:47.465265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465946+0000","last_change":"2024-04-26T19:59:45.286181+0000","last_active":"2024-04-26T19:59:47.465946+0000","last_peered":"2024-04-26T19:59:47.465946+0000","last_clean":"2024-04-26T19:59:47.465946+0000","last_became_active":"2024-04-26T19:59:45.285851+0000","last_became_peered":"2024-04-26T19:59:45.285851+0000","last_unstale":"2024-04-26T19:59:47.465946+0000","last_undegraded":"2024-04-26T19:59:47.465946+0000","last_fullsized":"2024-04-26T19:59:47.465946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463328+0000","last_change":"2024-04-26T19:59:45.286697+0000","last_active":"2024-04-26T19:59:47.463328+0000","last_peered":"2024-04-26T19:59:47.463328+0000","last_clean":"2024-04-26T19:59:47.463328+0000","last_became_active":"2024-04-26T19:59:45.286293+0000","last_became_peered":"2024-04-26T19:59:45.286293+0000","last_unstale":"2024-04-26T19:59:47.463328+0000","last_undegraded":"2024-04-26T19:59:47.463328+0000","last_fullsized":"2024-04-26T19:59:47.463328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464589+0000","last_change":"2024-04-26T19:59:46.294713+0000","last_active":"2024-04-26T19:59:47.464589+0000","last_peered":"2024-04-26T19:59:47.464589+0000","last_clean":"2024-04-26T19:59:47.464589+0000","last_became_active":"2024-04-26T19:59:46.294022+0000","last_became_peered":"2024-04-26T19:59:46.294022+0000","last_unstale":"2024-04-26T19:59:47.464589+0000","last_undegraded":"2024-04-26T19:59:47.464589+0000","last_fullsized":"2024-04-26T19:59:47.464589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464432+0000","last_change":"2024-04-26T19:59:45.284101+0000","last_active":"2024-04-26T19:59:47.464432+0000","last_peered":"2024-04-26T19:59:47.464432+0000","last_clean":"2024-04-26T19:59:47.464432+0000","last_became_active":"2024-04-26T19:59:45.283784+0000","last_became_peered":"2024-04-26T19:59:45.283784+0000","last_unstale":"2024-04-26T19:59:47.464432+0000","last_undegraded":"2024-04-26T19:59:47.464432+0000","last_fullsized":"2024-04-26T19:59:47.464432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293526+0000","last_change":"2024-04-26T19:59:46.296659+0000","last_active":"2024-04-26T19:59:47.293526+0000","last_peered":"2024-04-26T19:59:47.293526+0000","last_clean":"2024-04-26T19:59:47.293526+0000","last_became_active":"2024-04-26T19:59:46.296362+0000","last_became_peered":"2024-04-26T19:59:46.296362+0000","last_unstale":"2024-04-26T19:59:47.293526+0000","last_undegraded":"2024-04-26T19:59:47.293526+0000","last_fullsized":"2024-04-26T19:59:47.293526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468741+0000","last_change":"2024-04-26T19:59:45.299230+0000","last_active":"2024-04-26T19:59:47.468741+0000","last_peered":"2024-04-26T19:59:47.468741+0000","last_clean":"2024-04-26T19:59:47.468741+0000","last_became_active":"2024-04-26T19:59:45.298903+0000","last_became_peered":"2024-04-26T19:59:45.298903+0000","last_unstale":"2024-04-26T19:59:47.468741+0000","last_undegraded":"2024-04-26T19:59:47.468741+0000","last_fullsized":"2024-04-26T19:59:47.468741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467717+0000","last_change":"2024-04-26T19:59:46.306993+0000","last_active":"2024-04-26T19:59:47.467717+0000","last_peered":"2024-04-26T19:59:47.467717+0000","last_clean":"2024-04-26T19:59:47.467717+0000","last_became_active":"2024-04-26T19:59:46.305722+0000","last_became_peered":"2024-04-26T19:59:46.305722+0000","last_unstale":"2024-04-26T19:59:47.467717+0000","last_undegraded":"2024-04-26T19:59:47.467717+0000","last_fullsized":"2024-04-26T19:59:47.467717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467742+0000","last_change":"2024-04-26T19:59:45.297752+0000","last_active":"2024-04-26T19:59:47.467742+0000","last_peered":"2024-04-26T19:59:47.467742+0000","last_clean":"2024-04-26T19:59:47.467742+0000","last_became_active":"2024-04-26T19:59:45.297462+0000","last_became_peered":"2024-04-26T19:59:45.297462+0000","last_unstale":"2024-04-26T19:59:47.467742+0000","last_undegraded":"2024-04-26T19:59:47.467742+0000","last_fullsized":"2024-04-26T19:59:47.467742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464194+0000","last_change":"2024-04-26T19:59:46.290536+0000","last_active":"2024-04-26T19:59:47.464194+0000","last_peered":"2024-04-26T19:59:47.464194+0000","last_clean":"2024-04-26T19:59:47.464194+0000","last_became_active":"2024-04-26T19:59:46.289737+0000","last_became_peered":"2024-04-26T19:59:46.289737+0000","last_unstale":"2024-04-26T19:59:47.464194+0000","last_undegraded":"2024-04-26T19:59:47.464194+0000","last_fullsized":"2024-04-26T19:59:47.464194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294232+0000","last_change":"2024-04-26T19:59:45.293462+0000","last_active":"2024-04-26T19:59:47.294232+0000","last_peered":"2024-04-26T19:59:47.294232+0000","last_clean":"2024-04-26T19:59:47.294232+0000","last_became_active":"2024-04-26T19:59:45.292997+0000","last_became_peered":"2024-04-26T19:59:45.292997+0000","last_unstale":"2024-04-26T19:59:47.294232+0000","last_undegraded":"2024-04-26T19:59:47.294232+0000","last_fullsized":"2024-04-26T19:59:47.294232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298199+0000","last_change":"2024-04-26T19:59:46.307369+0000","last_active":"2024-04-26T19:59:47.298199+0000","last_peered":"2024-04-26T19:59:47.298199+0000","last_clean":"2024-04-26T19:59:47.298199+0000","last_became_active":"2024-04-26T19:59:46.306973+0000","last_became_peered":"2024-04-26T19:59:46.306973+0000","last_unstale":"2024-04-26T19:59:47.298199+0000","last_undegraded":"2024-04-26T19:59:47.298199+0000","last_fullsized":"2024-04-26T19:59:47.298199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469706+0000","last_change":"2024-04-26T19:59:45.292264+0000","last_active":"2024-04-26T19:59:47.469706+0000","last_peered":"2024-04-26T19:59:47.469706+0000","last_clean":"2024-04-26T19:59:47.469706+0000","last_became_active":"2024-04-26T19:59:45.290211+0000","last_became_peered":"2024-04-26T19:59:45.290211+0000","last_unstale":"2024-04-26T19:59:47.469706+0000","last_undegraded":"2024-04-26T19:59:47.469706+0000","last_fullsized":"2024-04-26T19:59:47.469706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467202+0000","last_change":"2024-04-26T19:59:46.294340+0000","last_active":"2024-04-26T19:59:47.467202+0000","last_peered":"2024-04-26T19:59:47.467202+0000","last_clean":"2024-04-26T19:59:47.467202+0000","last_became_active":"2024-04-26T19:59:46.294017+0000","last_became_peered":"2024-04-26T19:59:46.294017+0000","last_unstale":"2024-04-26T19:59:47.467202+0000","last_undegraded":"2024-04-26T19:59:47.467202+0000","last_fullsized":"2024-04-26T19:59:47.467202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465254+0000","last_change":"2024-04-26T19:59:45.285166+0000","last_active":"2024-04-26T19:59:47.465254+0000","last_peered":"2024-04-26T19:59:47.465254+0000","last_clean":"2024-04-26T19:59:47.465254+0000","last_became_active":"2024-04-26T19:59:45.284863+0000","last_became_peered":"2024-04-26T19:59:45.284863+0000","last_unstale":"2024-04-26T19:59:47.465254+0000","last_undegraded":"2024-04-26T19:59:47.465254+0000","last_fullsized":"2024-04-26T19:59:47.465254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464651+0000","last_change":"2024-04-26T19:59:46.296940+0000","last_active":"2024-04-26T19:59:47.464651+0000","last_peered":"2024-04-26T19:59:47.464651+0000","last_clean":"2024-04-26T19:59:47.464651+0000","last_became_active":"2024-04-26T19:59:46.296695+0000","last_became_peered":"2024-04-26T19:59:46.296695+0000","last_unstale":"2024-04-26T19:59:47.464651+0000","last_undegraded":"2024-04-26T19:59:47.464651+0000","last_fullsized":"2024-04-26T19:59:47.464651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466255+0000","last_change":"2024-04-26T19:59:45.288319+0000","last_active":"2024-04-26T19:59:47.466255+0000","last_peered":"2024-04-26T19:59:47.466255+0000","last_clean":"2024-04-26T19:59:47.466255+0000","last_became_active":"2024-04-26T19:59:45.287908+0000","last_became_peered":"2024-04-26T19:59:45.287908+0000","last_unstale":"2024-04-26T19:59:47.466255+0000","last_undegraded":"2024-04-26T19:59:47.466255+0000","last_fullsized":"2024-04-26T19:59:47.466255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469394+0000","last_change":"2024-04-26T19:59:46.314258+0000","last_active":"2024-04-26T19:59:47.469394+0000","last_peered":"2024-04-26T19:59:47.469394+0000","last_clean":"2024-04-26T19:59:47.469394+0000","last_became_active":"2024-04-26T19:59:46.313941+0000","last_became_peered":"2024-04-26T19:59:46.313941+0000","last_unstale":"2024-04-26T19:59:47.469394+0000","last_undegraded":"2024-04-26T19:59:47.469394+0000","last_fullsized":"2024-04-26T19:59:47.469394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466928+0000","last_change":"2024-04-26T19:59:45.292499+0000","last_active":"2024-04-26T19:59:47.466928+0000","last_peered":"2024-04-26T19:59:47.466928+0000","last_clean":"2024-04-26T19:59:47.466928+0000","last_became_active":"2024-04-26T19:59:45.288991+0000","last_became_peered":"2024-04-26T19:59:45.288991+0000","last_unstale":"2024-04-26T19:59:47.466928+0000","last_undegraded":"2024-04-26T19:59:47.466928+0000","last_fullsized":"2024-04-26T19:59:47.466928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465852+0000","last_change":"2024-04-26T19:59:46.283597+0000","last_active":"2024-04-26T19:59:47.465852+0000","last_peered":"2024-04-26T19:59:47.465852+0000","last_clean":"2024-04-26T19:59:47.465852+0000","last_became_active":"2024-04-26T19:59:46.283235+0000","last_became_peered":"2024-04-26T19:59:46.283235+0000","last_unstale":"2024-04-26T19:59:47.465852+0000","last_undegraded":"2024-04-26T19:59:47.465852+0000","last_fullsized":"2024-04-26T19:59:47.465852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297270+0000","last_change":"2024-04-26T19:59:45.276538+0000","last_active":"2024-04-26T19:59:47.297270+0000","last_peered":"2024-04-26T19:59:47.297270+0000","last_clean":"2024-04-26T19:59:47.297270+0000","last_became_active":"2024-04-26T19:59:45.276197+0000","last_became_peered":"2024-04-26T19:59:45.276197+0000","last_unstale":"2024-04-26T19:59:47.297270+0000","last_undegraded":"2024-04-26T19:59:47.297270+0000","last_fullsized":"2024-04-26T19:59:47.297270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466760+0000","last_change":"2024-04-26T19:59:46.314180+0000","last_active":"2024-04-26T19:59:47.466760+0000","last_peered":"2024-04-26T19:59:47.466760+0000","last_clean":"2024-04-26T19:59:47.466760+0000","last_became_active":"2024-04-26T19:59:46.313845+0000","last_became_peered":"2024-04-26T19:59:46.313845+0000","last_unstale":"2024-04-26T19:59:47.466760+0000","last_undegraded":"2024-04-26T19:59:47.466760+0000","last_fullsized":"2024-04-26T19:59:47.466760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298083+0000","last_change":"2024-04-26T19:59:45.282738+0000","last_active":"2024-04-26T19:59:47.298083+0000","last_peered":"2024-04-26T19:59:47.298083+0000","last_clean":"2024-04-26T19:59:47.298083+0000","last_became_active":"2024-04-26T19:59:45.281228+0000","last_became_peered":"2024-04-26T19:59:45.281228+0000","last_unstale":"2024-04-26T19:59:47.298083+0000","last_undegraded":"2024-04-26T19:59:47.298083+0000","last_fullsized":"2024-04-26T19:59:47.298083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465206+0000","last_change":"2024-04-26T19:59:46.314419+0000","last_active":"2024-04-26T19:59:47.465206+0000","last_peered":"2024-04-26T19:59:47.465206+0000","last_clean":"2024-04-26T19:59:47.465206+0000","last_became_active":"2024-04-26T19:59:46.313946+0000","last_became_peered":"2024-04-26T19:59:46.313946+0000","last_unstale":"2024-04-26T19:59:47.465206+0000","last_undegraded":"2024-04-26T19:59:47.465206+0000","last_fullsized":"2024-04-26T19:59:47.465206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297898+0000","last_change":"2024-04-26T19:59:45.266464+0000","last_active":"2024-04-26T19:59:47.297898+0000","last_peered":"2024-04-26T19:59:47.297898+0000","last_clean":"2024-04-26T19:59:47.297898+0000","last_became_active":"2024-04-26T19:59:45.265937+0000","last_became_peered":"2024-04-26T19:59:45.265937+0000","last_unstale":"2024-04-26T19:59:47.297898+0000","last_undegraded":"2024-04-26T19:59:47.297898+0000","last_fullsized":"2024-04-26T19:59:47.297898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:00:35.753658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465838+0000","last_change":"2024-04-26T19:59:46.314568+0000","last_active":"2024-04-26T19:59:47.465838+0000","last_peered":"2024-04-26T19:59:47.465838+0000","last_clean":"2024-04-26T19:59:47.465838+0000","last_became_active":"2024-04-26T19:59:46.313865+0000","last_became_peered":"2024-04-26T19:59:46.313865+0000","last_unstale":"2024-04-26T19:59:47.465838+0000","last_undegraded":"2024-04-26T19:59:47.465838+0000","last_fullsized":"2024-04-26T19:59:47.465838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294351+0000","last_change":"2024-04-26T19:59:45.281434+0000","last_active":"2024-04-26T19:59:47.294351+0000","last_peered":"2024-04-26T19:59:47.294351+0000","last_clean":"2024-04-26T19:59:47.294351+0000","last_became_active":"2024-04-26T19:59:45.280816+0000","last_became_peered":"2024-04-26T19:59:45.280816+0000","last_unstale":"2024-04-26T19:59:47.294351+0000","last_undegraded":"2024-04-26T19:59:47.294351+0000","last_fullsized":"2024-04-26T19:59:47.294351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466518+0000","last_change":"2024-04-26T19:59:46.295918+0000","last_active":"2024-04-26T19:59:47.466518+0000","last_peered":"2024-04-26T19:59:47.466518+0000","last_clean":"2024-04-26T19:59:47.466518+0000","last_became_active":"2024-04-26T19:59:46.295630+0000","last_became_peered":"2024-04-26T19:59:46.295630+0000","last_unstale":"2024-04-26T19:59:47.466518+0000","last_undegraded":"2024-04-26T19:59:47.466518+0000","last_fullsized":"2024-04-26T19:59:47.466518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297781+0000","last_change":"2024-04-26T19:59:45.288285+0000","last_active":"2024-04-26T19:59:47.297781+0000","last_peered":"2024-04-26T19:59:47.297781+0000","last_clean":"2024-04-26T19:59:47.297781+0000","last_became_active":"2024-04-26T19:59:45.287962+0000","last_became_peered":"2024-04-26T19:59:45.287962+0000","last_unstale":"2024-04-26T19:59:47.297781+0000","last_undegraded":"2024-04-26T19:59:47.297781+0000","last_fullsized":"2024-04-26T19:59:47.297781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467597+0000","last_change":"2024-04-26T19:59:46.310806+0000","last_active":"2024-04-26T19:59:47.467597+0000","last_peered":"2024-04-26T19:59:47.467597+0000","last_clean":"2024-04-26T19:59:47.467597+0000","last_became_active":"2024-04-26T19:59:46.310281+0000","last_became_peered":"2024-04-26T19:59:46.310281+0000","last_unstale":"2024-04-26T19:59:47.467597+0000","last_undegraded":"2024-04-26T19:59:47.467597+0000","last_fullsized":"2024-04-26T19:59:47.467597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464724+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T19:59:47.464724+0000","last_peered":"2024-04-26T19:59:47.464724+0000","last_clean":"2024-04-26T19:59:47.464724+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T19:59:47.464724+0000","last_undegraded":"2024-04-26T19:59:47.464724+0000","last_fullsized":"2024-04-26T19:59:47.464724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465285+0000","last_change":"2024-04-26T19:59:45.281392+0000","last_active":"2024-04-26T19:59:47.465285+0000","last_peered":"2024-04-26T19:59:47.465285+0000","last_clean":"2024-04-26T19:59:47.465285+0000","last_became_active":"2024-04-26T19:59:45.281041+0000","last_became_peered":"2024-04-26T19:59:45.281041+0000","last_unstale":"2024-04-26T19:59:47.465285+0000","last_undegraded":"2024-04-26T19:59:47.465285+0000","last_fullsized":"2024-04-26T19:59:47.465285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465458+0000","last_change":"2024-04-26T19:59:46.294932+0000","last_active":"2024-04-26T19:59:47.465458+0000","last_peered":"2024-04-26T19:59:47.465458+0000","last_clean":"2024-04-26T19:59:47.465458+0000","last_became_active":"2024-04-26T19:59:46.294670+0000","last_became_peered":"2024-04-26T19:59:46.294670+0000","last_unstale":"2024-04-26T19:59:47.465458+0000","last_undegraded":"2024-04-26T19:59:47.465458+0000","last_fullsized":"2024-04-26T19:59:47.465458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464419+0000","last_change":"2024-04-26T19:59:45.299670+0000","last_active":"2024-04-26T19:59:47.464419+0000","last_peered":"2024-04-26T19:59:47.464419+0000","last_clean":"2024-04-26T19:59:47.464419+0000","last_became_active":"2024-04-26T19:59:45.299344+0000","last_became_peered":"2024-04-26T19:59:45.299344+0000","last_unstale":"2024-04-26T19:59:47.464419+0000","last_undegraded":"2024-04-26T19:59:47.464419+0000","last_fullsized":"2024-04-26T19:59:47.464419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465122+0000","last_change":"2024-04-26T19:59:46.300383+0000","last_active":"2024-04-26T19:59:47.465122+0000","last_peered":"2024-04-26T19:59:47.465122+0000","last_clean":"2024-04-26T19:59:47.465122+0000","last_became_active":"2024-04-26T19:59:46.299990+0000","last_became_peered":"2024-04-26T19:59:46.299990+0000","last_unstale":"2024-04-26T19:59:47.465122+0000","last_undegraded":"2024-04-26T19:59:47.465122+0000","last_fullsized":"2024-04-26T19:59:47.465122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297425+0000","last_change":"2024-04-26T19:59:45.270294+0000","last_active":"2024-04-26T19:59:47.297425+0000","last_peered":"2024-04-26T19:59:47.297425+0000","last_clean":"2024-04-26T19:59:47.297425+0000","last_became_active":"2024-04-26T19:59:45.269847+0000","last_became_peered":"2024-04-26T19:59:45.269847+0000","last_unstale":"2024-04-26T19:59:47.297425+0000","last_undegraded":"2024-04-26T19:59:47.297425+0000","last_fullsized":"2024-04-26T19:59:47.297425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294086+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.294086+0000","last_peered":"2024-04-26T19:59:47.294086+0000","last_clean":"2024-04-26T19:59:47.294086+0000","last_became_active":"2024-04-26T19:59:46.290208+0000","last_became_peered":"2024-04-26T19:59:46.290208+0000","last_unstale":"2024-04-26T19:59:47.294086+0000","last_undegraded":"2024-04-26T19:59:47.294086+0000","last_fullsized":"2024-04-26T19:59:47.294086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466469+0000","last_change":"2024-04-26T19:59:45.292265+0000","last_active":"2024-04-26T19:59:47.466469+0000","last_peered":"2024-04-26T19:59:47.466469+0000","last_clean":"2024-04-26T19:59:47.466469+0000","last_became_active":"2024-04-26T19:59:45.291785+0000","last_became_peered":"2024-04-26T19:59:45.291785+0000","last_unstale":"2024-04-26T19:59:47.466469+0000","last_undegraded":"2024-04-26T19:59:47.466469+0000","last_fullsized":"2024-04-26T19:59:47.466469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297995+0000","last_change":"2024-04-26T19:59:46.294373+0000","last_active":"2024-04-26T19:59:47.297995+0000","last_peered":"2024-04-26T19:59:47.297995+0000","last_clean":"2024-04-26T19:59:47.297995+0000","last_became_active":"2024-04-26T19:59:46.292939+0000","last_became_peered":"2024-04-26T19:59:46.292939+0000","last_unstale":"2024-04-26T19:59:47.297995+0000","last_undegraded":"2024-04-26T19:59:47.297995+0000","last_fullsized":"2024-04-26T19:59:47.297995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294407+0000","last_change":"2024-04-26T19:59:45.286062+0000","last_active":"2024-04-26T19:59:47.294407+0000","last_peered":"2024-04-26T19:59:47.294407+0000","last_clean":"2024-04-26T19:59:47.294407+0000","last_became_active":"2024-04-26T19:59:45.282680+0000","last_became_peered":"2024-04-26T19:59:45.282680+0000","last_unstale":"2024-04-26T19:59:47.294407+0000","last_undegraded":"2024-04-26T19:59:47.294407+0000","last_fullsized":"2024-04-26T19:59:47.294407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467709+0000","last_change":"2024-04-26T19:59:46.303855+0000","last_active":"2024-04-26T19:59:47.467709+0000","last_peered":"2024-04-26T19:59:47.467709+0000","last_clean":"2024-04-26T19:59:47.467709+0000","last_became_active":"2024-04-26T19:59:46.298323+0000","last_became_peered":"2024-04-26T19:59:46.298323+0000","last_unstale":"2024-04-26T19:59:47.467709+0000","last_undegraded":"2024-04-26T19:59:47.467709+0000","last_fullsized":"2024-04-26T19:59:47.467709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468317+0000","last_change":"2024-04-26T19:59:45.302616+0000","last_active":"2024-04-26T19:59:47.468317+0000","last_peered":"2024-04-26T19:59:47.468317+0000","last_clean":"2024-04-26T19:59:47.468317+0000","last_became_active":"2024-04-26T19:59:45.302164+0000","last_became_peered":"2024-04-26T19:59:45.302164+0000","last_unstale":"2024-04-26T19:59:47.468317+0000","last_undegraded":"2024-04-26T19:59:47.468317+0000","last_fullsized":"2024-04-26T19:59:47.468317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294273+0000","last_change":"2024-04-26T19:59:46.290753+0000","last_active":"2024-04-26T19:59:47.294273+0000","last_peered":"2024-04-26T19:59:47.294273+0000","last_clean":"2024-04-26T19:59:47.294273+0000","last_became_active":"2024-04-26T19:59:46.290477+0000","last_became_peered":"2024-04-26T19:59:46.290477+0000","last_unstale":"2024-04-26T19:59:47.294273+0000","last_undegraded":"2024-04-26T19:59:47.294273+0000","last_fullsized":"2024-04-26T19:59:47.294273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294611+0000","last_change":"2024-04-26T19:59:45.297751+0000","last_active":"2024-04-26T19:59:47.294611+0000","last_peered":"2024-04-26T19:59:47.294611+0000","last_clean":"2024-04-26T19:59:47.294611+0000","last_became_active":"2024-04-26T19:59:45.297426+0000","last_became_peered":"2024-04-26T19:59:45.297426+0000","last_unstale":"2024-04-26T19:59:47.294611+0000","last_undegraded":"2024-04-26T19:59:47.294611+0000","last_fullsized":"2024-04-26T19:59:47.294611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465393+0000","last_change":"2024-04-26T19:59:46.298839+0000","last_active":"2024-04-26T19:59:47.465393+0000","last_peered":"2024-04-26T19:59:47.465393+0000","last_clean":"2024-04-26T19:59:47.465393+0000","last_became_active":"2024-04-26T19:59:46.298552+0000","last_became_peered":"2024-04-26T19:59:46.298552+0000","last_unstale":"2024-04-26T19:59:47.465393+0000","last_undegraded":"2024-04-26T19:59:47.465393+0000","last_fullsized":"2024-04-26T19:59:47.465393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468933+0000","last_change":"2024-04-26T19:59:45.291668+0000","last_active":"2024-04-26T19:59:47.468933+0000","last_peered":"2024-04-26T19:59:47.468933+0000","last_clean":"2024-04-26T19:59:47.468933+0000","last_became_active":"2024-04-26T19:59:45.291333+0000","last_became_peered":"2024-04-26T19:59:45.291333+0000","last_unstale":"2024-04-26T19:59:47.468933+0000","last_undegraded":"2024-04-26T19:59:47.468933+0000","last_fullsized":"2024-04-26T19:59:47.468933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465754+0000","last_change":"2024-04-26T19:59:46.276488+0000","last_active":"2024-04-26T19:59:47.465754+0000","last_peered":"2024-04-26T19:59:47.465754+0000","last_clean":"2024-04-26T19:59:47.465754+0000","last_became_active":"2024-04-26T19:59:46.274349+0000","last_became_peered":"2024-04-26T19:59:46.274349+0000","last_unstale":"2024-04-26T19:59:47.465754+0000","last_undegraded":"2024-04-26T19:59:47.465754+0000","last_fullsized":"2024-04-26T19:59:47.465754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464100+0000","last_change":"2024-04-26T19:59:45.300210+0000","last_active":"2024-04-26T19:59:47.464100+0000","last_peered":"2024-04-26T19:59:47.464100+0000","last_clean":"2024-04-26T19:59:47.464100+0000","last_became_active":"2024-04-26T19:59:45.299954+0000","last_became_peered":"2024-04-26T19:59:45.299954+0000","last_unstale":"2024-04-26T19:59:47.464100+0000","last_undegraded":"2024-04-26T19:59:47.464100+0000","last_fullsized":"2024-04-26T19:59:47.464100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297802+0000","last_change":"2024-04-26T19:59:46.299193+0000","last_active":"2024-04-26T19:59:47.297802+0000","last_peered":"2024-04-26T19:59:47.297802+0000","last_clean":"2024-04-26T19:59:47.297802+0000","last_became_active":"2024-04-26T19:59:46.298053+0000","last_became_peered":"2024-04-26T19:59:46.298053+0000","last_unstale":"2024-04-26T19:59:47.297802+0000","last_undegraded":"2024-04-26T19:59:47.297802+0000","last_fullsized":"2024-04-26T19:59:47.297802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:18.579103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293761+0000","last_change":"2024-04-26T19:59:45.290679+0000","last_active":"2024-04-26T19:59:47.293761+0000","last_peered":"2024-04-26T19:59:47.293761+0000","last_clean":"2024-04-26T19:59:47.293761+0000","last_became_active":"2024-04-26T19:59:45.290411+0000","last_became_peered":"2024-04-26T19:59:45.290411+0000","last_unstale":"2024-04-26T19:59:47.293761+0000","last_undegraded":"2024-04-26T19:59:47.293761+0000","last_fullsized":"2024-04-26T19:59:47.293761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464967+0000","last_change":"2024-04-26T19:59:46.288442+0000","last_active":"2024-04-26T19:59:47.464967+0000","last_peered":"2024-04-26T19:59:47.464967+0000","last_clean":"2024-04-26T19:59:47.464967+0000","last_became_active":"2024-04-26T19:59:46.288239+0000","last_became_peered":"2024-04-26T19:59:46.288239+0000","last_unstale":"2024-04-26T19:59:47.464967+0000","last_undegraded":"2024-04-26T19:59:47.464967+0000","last_fullsized":"2024-04-26T19:59:47.464967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294034+0000","last_change":"2024-04-26T19:59:45.280986+0000","last_active":"2024-04-26T19:59:47.294034+0000","last_peered":"2024-04-26T19:59:47.294034+0000","last_clean":"2024-04-26T19:59:47.294034+0000","last_became_active":"2024-04-26T19:59:45.280606+0000","last_became_peered":"2024-04-26T19:59:45.280606+0000","last_unstale":"2024-04-26T19:59:47.294034+0000","last_undegraded":"2024-04-26T19:59:47.294034+0000","last_fullsized":"2024-04-26T19:59:47.294034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464007+0000","last_change":"2024-04-26T19:59:46.296268+0000","last_active":"2024-04-26T19:59:47.464007+0000","last_peered":"2024-04-26T19:59:47.464007+0000","last_clean":"2024-04-26T19:59:47.464007+0000","last_became_active":"2024-04-26T19:59:46.294912+0000","last_became_peered":"2024-04-26T19:59:46.294912+0000","last_unstale":"2024-04-26T19:59:47.464007+0000","last_undegraded":"2024-04-26T19:59:47.464007+0000","last_fullsized":"2024-04-26T19:59:47.464007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469551+0000","last_change":"2024-04-26T19:59:45.299596+0000","last_active":"2024-04-26T19:59:47.469551+0000","last_peered":"2024-04-26T19:59:47.469551+0000","last_clean":"2024-04-26T19:59:47.469551+0000","last_became_active":"2024-04-26T19:59:45.299284+0000","last_became_peered":"2024-04-26T19:59:45.299284+0000","last_unstale":"2024-04-26T19:59:47.469551+0000","last_undegraded":"2024-04-26T19:59:47.469551+0000","last_fullsized":"2024-04-26T19:59:47.469551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465783+0000","last_change":"2024-04-26T19:59:46.294465+0000","last_active":"2024-04-26T19:59:47.465783+0000","last_peered":"2024-04-26T19:59:47.465783+0000","last_clean":"2024-04-26T19:59:47.465783+0000","last_became_active":"2024-04-26T19:59:46.294138+0000","last_became_peered":"2024-04-26T19:59:46.294138+0000","last_unstale":"2024-04-26T19:59:47.465783+0000","last_undegraded":"2024-04-26T19:59:47.465783+0000","last_fullsized":"2024-04-26T19:59:47.465783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467698+0000","last_change":"2024-04-26T19:59:46.304096+0000","last_active":"2024-04-26T19:59:47.467698+0000","last_peered":"2024-04-26T19:59:47.467698+0000","last_clean":"2024-04-26T19:59:47.467698+0000","last_became_active":"2024-04-26T19:59:46.299682+0000","last_became_peered":"2024-04-26T19:59:46.299682+0000","last_unstale":"2024-04-26T19:59:47.467698+0000","last_undegraded":"2024-04-26T19:59:47.467698+0000","last_fullsized":"2024-04-26T19:59:47.467698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467128+0000","last_change":"2024-04-26T19:59:45.302197+0000","last_active":"2024-04-26T19:59:47.467128+0000","last_peered":"2024-04-26T19:59:47.467128+0000","last_clean":"2024-04-26T19:59:47.467128+0000","last_became_active":"2024-04-26T19:59:45.301818+0000","last_became_peered":"2024-04-26T19:59:45.301818+0000","last_unstale":"2024-04-26T19:59:47.467128+0000","last_undegraded":"2024-04-26T19:59:47.467128+0000","last_fullsized":"2024-04-26T19:59:47.467128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:43.563741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466068+0000","last_change":"2024-04-26T19:59:46.309844+0000","last_active":"2024-04-26T19:59:47.466068+0000","last_peered":"2024-04-26T19:59:47.466068+0000","last_clean":"2024-04-26T19:59:47.466068+0000","last_became_active":"2024-04-26T19:59:46.309519+0000","last_became_peered":"2024-04-26T19:59:46.309519+0000","last_unstale":"2024-04-26T19:59:47.466068+0000","last_undegraded":"2024-04-26T19:59:47.466068+0000","last_fullsized":"2024-04-26T19:59:47.466068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467758+0000","last_change":"2024-04-26T19:59:45.301972+0000","last_active":"2024-04-26T19:59:47.467758+0000","last_peered":"2024-04-26T19:59:47.467758+0000","last_clean":"2024-04-26T19:59:47.467758+0000","last_became_active":"2024-04-26T19:59:45.301689+0000","last_became_peered":"2024-04-26T19:59:45.301689+0000","last_unstale":"2024-04-26T19:59:47.467758+0000","last_undegraded":"2024-04-26T19:59:47.467758+0000","last_fullsized":"2024-04-26T19:59:47.467758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463870+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.463870+0000","last_peered":"2024-04-26T19:59:47.463870+0000","last_clean":"2024-04-26T19:59:47.463870+0000","last_became_active":"2024-04-26T19:59:46.290172+0000","last_became_peered":"2024-04-26T19:59:46.290172+0000","last_unstale":"2024-04-26T19:59:47.463870+0000","last_undegraded":"2024-04-26T19:59:47.463870+0000","last_fullsized":"2024-04-26T19:59:47.463870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464753+0000","last_change":"2024-04-26T19:59:45.287597+0000","last_active":"2024-04-26T19:59:47.464753+0000","last_peered":"2024-04-26T19:59:47.464753+0000","last_clean":"2024-04-26T19:59:47.464753+0000","last_became_active":"2024-04-26T19:59:45.287273+0000","last_became_peered":"2024-04-26T19:59:45.287273+0000","last_unstale":"2024-04-26T19:59:47.464753+0000","last_undegraded":"2024-04-26T19:59:47.464753+0000","last_fullsized":"2024-04-26T19:59:47.464753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463701+0000","last_change":"2024-04-26T19:59:46.301596+0000","last_active":"2024-04-26T19:59:47.463701+0000","last_peered":"2024-04-26T19:59:47.463701+0000","last_clean":"2024-04-26T19:59:47.463701+0000","last_became_active":"2024-04-26T19:59:46.292219+0000","last_became_peered":"2024-04-26T19:59:46.292219+0000","last_unstale":"2024-04-26T19:59:47.463701+0000","last_undegraded":"2024-04-26T19:59:47.463701+0000","last_fullsized":"2024-04-26T19:59:47.463701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464191+0000","last_change":"2024-04-26T19:59:45.293237+0000","last_active":"2024-04-26T19:59:47.464191+0000","last_peered":"2024-04-26T19:59:47.464191+0000","last_clean":"2024-04-26T19:59:47.464191+0000","last_became_active":"2024-04-26T19:59:45.292699+0000","last_became_peered":"2024-04-26T19:59:45.292699+0000","last_unstale":"2024-04-26T19:59:47.464191+0000","last_undegraded":"2024-04-26T19:59:47.464191+0000","last_fullsized":"2024-04-26T19:59:47.464191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:59:51.194 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T19:59:51.194 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T19:59:51.194 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T19:59:51.195 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T19:59:51.195 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T19:59:51.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.424+0000 7f4253800640 1 Processor -- start 2024-04-26T19:59:51.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.424+0000 7f4253800640 1 -- start start 2024-04-26T19:59:51.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.425+0000 7f4253800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f424c15cdf0 0x7f424c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.425+0000 7f4253800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f424c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.425+0000 7f4253800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424c0af9a0 con 0x7f424c0cd650 2024-04-26T19:59:51.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.425+0000 7f4253800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f424c0ae2c0 con 0x7f424c15cdf0 2024-04-26T19:59:51.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.425+0000 7f4253800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f424c0b05b0 con 0x7f424c06ce80 2024-04-26T19:59:51.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f424be00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f424c0cd650 0x7f424c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52748/0 (socket says 172.21.15.88:52748) 2024-04-26T19:59:51.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f424be00640 1 -- 172.21.15.88:0/2856450900 learned_addr learned my addr 172.21.15.88:0/2856450900 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:51.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f4251800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f424c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f4250e00640 1 --2- 172.21.15.88:0/2856450900 >> v2:172.21.15.132:3300/0 conn(0x7f424c15cdf0 0x7f424c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108613698 0 0) 0x7f424c0af9a0 con 0x7f424c0cd650 2024-04-26T19:59:51.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.426+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4230003520 con 0x7f424c0cd650 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2685526647 0 0) 0x7f4230003520 con 0x7f424c0cd650 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 msgr2=0x7f424c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f424c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 >> v2:172.21.15.132:3300/0 conn(0x7f424c15cdf0 msgr2=0x7f424c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 --2- 172.21.15.88:0/2856450900 >> v2:172.21.15.132:3300/0 conn(0x7f424c15cdf0 0x7f424c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c0afc80 con 0x7f424c0cd650 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f42380019f0 con 0x7f424c0cd650 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 >> v1:172.21.15.88:6789/0 conn(0x7f424c0cd650 legacy=0x7f424c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42300039b0 0x7f4230003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f4230004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f42300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f424c0b05b0 con 0x7f4230004d40 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f423000b9c0 con 0x7f42300039b0 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f423000bb40 con 0x7f424c06ce80 2024-04-26T19:59:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f4251800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f42300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.427+0000 7f424be00640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f4230004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.428+0000 7f4251800640 1 -- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 msgr2=0x7f4230004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.428+0000 7f4251800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c06ce80 0x7f4230004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.428+0000 7f4251800640 1 -- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42300039b0 msgr2=0x7f4230003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.428+0000 7f4251800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42300039b0 0x7f4230003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.428+0000 7f4251800640 1 -- 172.21.15.88:0/2856450900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4230003520 con 0x7f4230004d40 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4251800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f42300051c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4244009ac0 tx=0x7f4230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fc589044470bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244036070 con 0x7f4230004d40 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4244031400 con 0x7f4230004d40 2024-04-26T19:59:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f424b400640 1 -- 172.21.15.88:0/2856450900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244009690 con 0x7f4230004d40 2024-04-26T19:59:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4253800640 1 -- 172.21.15.88:0/2856450900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 msgr2=0x7f42300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4253800640 1 --2- 172.21.15.88:0/2856450900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f42300051c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4244009ac0 tx=0x7f4230005830 comp rx=0 tx=0).stop 2024-04-26T19:59:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4251800640 1 -- 172.21.15.88:0/2856450900 reap_dead start 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4253800640 1 -- 172.21.15.88:0/2856450900 shutdown_connections 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.429+0000 7f4253800640 1 -- 172.21.15.88:0/2856450900 >> 172.21.15.88:0/2856450900 conn(0x7f424c0b2d50 msgr2=0x7f424c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.430+0000 7f4253800640 1 -- 172.21.15.88:0/2856450900 shutdown_connections 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.430+0000 7f4253800640 1 -- 172.21.15.88:0/2856450900 wait complete. 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.431+0000 7f4253800640 1 Processor -- start 2024-04-26T19:59:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.431+0000 7f4253800640 1 -- start start 2024-04-26T19:59:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f424c06ee30 0x7f424c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 0x7f424c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f424c15d1d0 con 0x7f4230004d40 2024-04-26T19:59:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f424c0b0240 con 0x7f424c06ee30 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.432+0000 7f4253800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f424c15d350 con 0x7f424c15bd10 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44912/0 (socket says 172.21.15.88:44912) 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 -- 172.21.15.88:0/777338994 learned_addr learned my addr 172.21.15.88:0/777338994 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4251800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 0x7f424c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 -- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 msgr2=0x7f424c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f424be00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f424c06ee30 0x7f424c15b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 0x7f424c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 -- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f424c06ee30 msgr2=0x7f424c15b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f424c06ee30 0x7f424c15b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 -- 172.21.15.88:0/777338994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c06f2b0 con 0x7f4230004d40 2024-04-26T19:59:51.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4251800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 0x7f424c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.433+0000 7f4250e00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e7c0 tx=0x7f423c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.434+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c017070 con 0x7f4230004d40 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.434+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423c00c940 con 0x7f4230004d40 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.434+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c00cd30 con 0x7f4230004d40 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.434+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c0cf700 con 0x7f4230004d40 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.434+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c0cfc80 con 0x7f4230004d40 2024-04-26T19:59:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.435+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424c069540 con 0x7f4230004d40 2024-04-26T19:59:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.436+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f423c01a070 con 0x7f4230004d40 2024-04-26T19:59:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.437+0000 7f4248c00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 0x7f422c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.437+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f423c08b9f0 con 0x7f4230004d40 2024-04-26T19:59:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.437+0000 7f424be00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 0x7f422c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.438+0000 7f424be00640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 0x7f422c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42380013d0 tx=0x7f4238001360 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.441+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f424c06f2b0 con 0x7f4230004d40 2024-04-26T19:59:51.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.683+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f424c15c0f0 con 0x7f4230004d40 2024-04-26T19:59:51.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.691+0000 7f4248c00640 1 -- 172.21.15.88:0/777338994 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174133 (secure 0 0 0) 0x7f423c059eb0 con 0x7f4230004d40 2024-04-26T19:59:51.695 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:51.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.694+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 msgr2=0x7f422c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.694+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 0x7f422c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42380013d0 tx=0x7f4238001360 comp rx=0 tx=0).stop 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.694+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 msgr2=0x7f424c146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.694+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e7c0 tx=0x7f423c00ec80 comp rx=0 tx=0).stop 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 shutdown_connections 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f422c06c8f0 0x7f422c06edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f424c15bd10 0x7f424c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f424c06ee30 0x7f424c15b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 --2- 172.21.15.88:0/777338994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4230004d40 0x7f424c146720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 >> 172.21.15.88:0/777338994 conn(0x7f424c0b2d50 msgr2=0x7f424c0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:51.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 shutdown_connections 2024-04-26T19:59:51.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.695+0000 7f4253800640 1 -- 172.21.15.88:0/777338994 wait complete. 2024-04-26T19:59:51.712 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":318043982},{"type":"v1","addr":"172.21.15.88:6833","nonce":318043982}]},"active_addr":"172.21.15.88:6833/318043982","active_change":"2024-04-26T19:59:26.916932+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":1838586336}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":2390660440}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":482721619}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3181067675}]}]} 2024-04-26T19:59:51.715 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T19:59:51.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T19:59:51.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:51.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.903+0000 7f61d4e00640 1 Processor -- start 2024-04-26T19:59:51.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.903+0000 7f61d4e00640 1 -- start start 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.904+0000 7f61d4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 0x7f61d00b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61d4e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61d00d0e70 0x7f61d00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61d4e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d0146a60 con 0x7f61d0160930 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61d4e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f61d006e970 con 0x7f61d00d0e70 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61d4e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f61d00d17f0 con 0x7f61d0146680 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61d00d0e70 0x7f61d00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f61d00d0e70 0x7f61d00d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44922/0 (socket says 172.21.15.88:44922) 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 -- 172.21.15.88:0/505791183 learned_addr learned my addr 172.21.15.88:0/505791183 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61cf400640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 0x7f61d00b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 -- 172.21.15.88:0/505791183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 msgr2=0x7f61d00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 0x7f61d00b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 -- 172.21.15.88:0/505791183 >> v1:172.21.15.132:6789/0 conn(0x7f61d0160930 legacy=0x7f61d0160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:51.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61c6a00640 1 -- 172.21.15.88:0/505791183 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d006dd60 con 0x7f61d00d0e70 2024-04-26T19:59:51.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.905+0000 7f61cf400640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 0x7f61d00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.906+0000 7f61c6a00640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d00d0e70 0x7f61d00d12b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009870 tx=0x7f61b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324813c96af95dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.906+0000 7f61cea00640 1 -- 172.21.15.88:0/505791183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8037070 con 0x7f61d00d0e70 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61cea00640 1 -- 172.21.15.88:0/505791183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8032440 con 0x7f61d00d0e70 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61cea00640 1 -- 172.21.15.88:0/505791183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8032810 con 0x7f61d00d0e70 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 -- 172.21.15.88:0/505791183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d00d0e70 msgr2=0x7f61d00d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d00d0e70 0x7f61d00d12b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009870 tx=0x7f61b802a210 comp rx=0 tx=0).stop 2024-04-26T19:59:51.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 -- 172.21.15.88:0/505791183 shutdown_connections 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d00d0e70 0x7f61d00d12b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 --2- 172.21.15.88:0/505791183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0146680 0x7f61d00b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 -- 172.21.15.88:0/505791183 >> 172.21.15.88:0/505791183 conn(0x7f61d006c740 msgr2=0x7f61d006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 -- 172.21.15.88:0/505791183 shutdown_connections 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.907+0000 7f61d4e00640 1 -- 172.21.15.88:0/505791183 wait complete. 2024-04-26T19:59:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.909+0000 7f61d4e00640 1 Processor -- start 2024-04-26T19:59:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.909+0000 7f61d4e00640 1 -- start start 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61d00d0e70 0x7f61d015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0160930 0x7f61d0143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f61d006e600 con 0x7f61d0146680 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f61d006e320 con 0x7f61d00d0e70 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61d4e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f61d00cd6a0 con 0x7f61d0160930 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44930/0 (socket says 172.21.15.88:44930) 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61c6a00640 1 -- 172.21.15.88:0/924022050 learned_addr learned my addr 172.21.15.88:0/924022050 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61cfe00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0160930 0x7f61d0143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.910+0000 7f61c6a00640 1 -- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0160930 msgr2=0x7f61d0143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c6a00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0160930 0x7f61d0143b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c6a00640 1 -- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61d00d0e70 msgr2=0x7f61d015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c6a00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61d00d0e70 0x7f61d015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c6a00640 1 -- 172.21.15.88:0/924022050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d0142d60 con 0x7f61d0146680 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c6a00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000c00 tx=0x7f61b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8037040 con 0x7f61d0146680 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d0143040 con 0x7f61d0146680 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d0143600 con 0x7f61d0146680 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b8042070 con 0x7f61d0146680 2024-04-26T19:59:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.911+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8030b70 con 0x7f61d0146680 2024-04-26T19:59:51.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.912+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a8005290 con 0x7f61d0146680 2024-04-26T19:59:51.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.913+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61b8030d10 con 0x7f61d0146680 2024-04-26T19:59:51.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.915+0000 7f61c7e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:51.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.915+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61b80ac7a0 con 0x7f61d0146680 2024-04-26T19:59:51.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.915+0000 7f61cf400640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:51.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.916+0000 7f61cf400640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 0x7f61ac06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006fd0 tx=0x7f61c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:51.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:51.920+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61b8045360 con 0x7f61d0146680 2024-04-26T19:59:52.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.106+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61a8005580 con 0x7f61d0146680 2024-04-26T19:59:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.109+0000 7f61c7e00640 1 -- 172.21.15.88:0/924022050 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f61b807abc0 con 0x7f61d0146680 2024-04-26T19:59:52.111 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:52.111 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 msgr2=0x7f61ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 0x7f61ac06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006fd0 tx=0x7f61c0008040 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 msgr2=0x7f61d00caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f61b8000c00 tx=0x7f61b80046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 shutdown_connections 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f61d0160930 0x7f61d0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61d0146680 0x7f61d00caef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 --2- 172.21.15.88:0/924022050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61d00d0e70 0x7f61d015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 >> 172.21.15.88:0/924022050 conn(0x7f61d006c740 msgr2=0x7f61d00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 shutdown_connections 2024-04-26T19:59:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.112+0000 7f61d4e00640 1 -- 172.21.15.88:0/924022050 wait complete. 2024-04-26T19:59:52.133 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T19:59:52.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:52.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.323+0000 7fc5a7e00640 1 Processor -- start 2024-04-26T19:59:52.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.323+0000 7fc5a7e00640 1 -- start start 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5a8145570 0x7fc5a80d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc5a8073ed0 0x7fc5a80d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a8073110 con 0x7fc5a815f940 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a7e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a80cef40 con 0x7fc5a8145570 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a81606b0 con 0x7fc5a8073ed0 2024-04-26T19:59:52.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5a8145570 0x7fc5a80d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5a8145570 0x7fc5a80d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44944/0 (socket says 172.21.15.88:44944) 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc5a6a00640 1 -- 172.21.15.88:0/2426437006 learned_addr learned my addr 172.21.15.88:0/2426437006 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.325+0000 7fc59fe00640 1 --2- 172.21.15.88:0/2426437006 >> v2:172.21.15.132:3300/0 conn(0x7fc5a8073ed0 0x7fc5a80d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.326+0000 7fc5a6a00640 1 -- 172.21.15.88:0/2426437006 >> v2:172.21.15.132:3300/0 conn(0x7fc5a8073ed0 msgr2=0x7fc5a80d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.326+0000 7fc5a6a00640 1 --2- 172.21.15.88:0/2426437006 >> v2:172.21.15.132:3300/0 conn(0x7fc5a8073ed0 0x7fc5a80d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.326+0000 7fc5a6a00640 1 -- 172.21.15.88:0/2426437006 >> v1:172.21.15.132:6789/0 conn(0x7fc5a815f940 legacy=0x7fc5a815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.326+0000 7fc5a6a00640 1 -- 172.21.15.88:0/2426437006 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80733f0 con 0x7fc5a8145570 2024-04-26T19:59:52.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.327+0000 7fc5a6a00640 1 --2- 172.21.15.88:0/2426437006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a8145570 0x7fc5a80d0e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8072d20 tx=0x7fc59802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2943f2a1b2d0470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.327+0000 7fc5a6000640 1 -- 172.21.15.88:0/2426437006 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598037070 con 0x7fc5a8145570 2024-04-26T19:59:52.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.327+0000 7fc5a6000640 1 -- 172.21.15.88:0/2426437006 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc598032440 con 0x7fc5a8145570 2024-04-26T19:59:52.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.327+0000 7fc5a6000640 1 -- 172.21.15.88:0/2426437006 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598032810 con 0x7fc5a8145570 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 -- 172.21.15.88:0/2426437006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a8145570 msgr2=0x7fc5a80d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/2426437006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a8145570 0x7fc5a80d0e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8072d20 tx=0x7fc59802a1a0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 -- 172.21.15.88:0/2426437006 shutdown_connections 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/2426437006 >> v2:172.21.15.132:3300/0 conn(0x7fc5a8073ed0 0x7fc5a80d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/2426437006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a8145570 0x7fc5a80d0e70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 -- 172.21.15.88:0/2426437006 >> 172.21.15.88:0/2426437006 conn(0x7fc5a806c740 msgr2=0x7fc5a806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 -- 172.21.15.88:0/2426437006 shutdown_connections 2024-04-26T19:59:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.328+0000 7fc5a7e00640 1 -- 172.21.15.88:0/2426437006 wait complete. 2024-04-26T19:59:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.330+0000 7fc5a7e00640 1 Processor -- start 2024-04-26T19:59:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.330+0000 7fc5a7e00640 1 -- start start 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8073ed0 0x7fc5a806de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 0x7fc5a8168440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a8145950 con 0x7fc5a815f940 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80739b0 con 0x7fc5a8073ed0 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80cdf30 con 0x7fc5a8145570 2024-04-26T19:59:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44950/0 (socket says 172.21.15.88:44950) 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7400640 1 -- 172.21.15.88:0/525428363 learned_addr learned my addr 172.21.15.88:0/525428363 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc59fe00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 0x7fc5a8168440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7400640 1 -- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 msgr2=0x7fc5a8168440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.331+0000 7fc5a7400640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 0x7fc5a8168440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7400640 1 -- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8073ed0 msgr2=0x7fc5a806de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7400640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8073ed0 0x7fc5a806de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7400640 1 -- 172.21.15.88:0/525428363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80ca1f0 con 0x7fc5a815f940 2024-04-26T19:59:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc59fe00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 0x7fc5a8168440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7400640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b9a0 tx=0x7fc5a000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0019070 con 0x7fc5a815f940 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a000cb40 con 0x7fc5a815f940 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0014440 con 0x7fc5a815f940 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a80ca4d0 con 0x7fc5a815f940 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.332+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a80caa90 con 0x7fc5a815f940 2024-04-26T19:59:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.333+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80693c0 con 0x7fc5a815f940 2024-04-26T19:59:52.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.335+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5a001c070 con 0x7fc5a815f940 2024-04-26T19:59:52.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.335+0000 7fc59ea00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 0x7fc58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.336+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc5a008b790 con 0x7fc5a815f940 2024-04-26T19:59:52.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.336+0000 7fc5a6a00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 0x7fc58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.336+0000 7fc5a6a00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 0x7fc58406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc598034000 tx=0x7fc59802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.340+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5a0059b20 con 0x7fc5a815f940 2024-04-26T19:59:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.529+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5a80b4850 con 0x7fc5a815f940 2024-04-26T19:59:52.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.532+0000 7fc59ea00640 1 -- 172.21.15.88:0/525428363 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7fc5a0059420 con 0x7fc5a815f940 2024-04-26T19:59:52.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:52.534 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:52.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.534+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 msgr2=0x7fc58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.534+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 0x7fc58406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc598034000 tx=0x7fc59802c000 comp rx=0 tx=0).stop 2024-04-26T19:59:52.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.534+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 msgr2=0x7fc5a80cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.534+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b9a0 tx=0x7fc5a000be60 comp rx=0 tx=0).stop 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 shutdown_connections 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5a815f940 0x7fc5a80cb290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc58406c8f0 0x7fc58406edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5a8145570 0x7fc5a8168440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 --2- 172.21.15.88:0/525428363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5a8073ed0 0x7fc5a806de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 >> 172.21.15.88:0/525428363 conn(0x7fc5a806c740 msgr2=0x7fc5a806ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 shutdown_connections 2024-04-26T19:59:52.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.535+0000 7fc5a7e00640 1 -- 172.21.15.88:0/525428363 wait complete. 2024-04-26T19:59:52.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T19:59:52.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T19:59:52.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T19:59:52.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T19:59:52.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T19:59:52.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T19:59:52.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T19:59:52.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T19:59:52.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.834+0000 7fbceb200640 1 Processor -- start 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.835+0000 7fbceb200640 1 -- start start 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbceb200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbce404fa90 0x7fbce404fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbceb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce4054270 0x7fbce40389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbceb200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce40503b0 con 0x7fbce40df420 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbceb200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fbce40068a0 con 0x7fbce404fa90 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbceb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbce4006af0 con 0x7fbce4054270 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbce9400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbce40df420 0x7fbce40382c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48090/0 (socket says 172.21.15.88:48090) 2024-04-26T19:59:52.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.837+0000 7fbce9400640 1 -- 172.21.15.88:0/3687760836 learned_addr learned my addr 172.21.15.88:0/3687760836 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.838+0000 7fbce9e00640 1 --2- 172.21.15.88:0/3687760836 >> v2:172.21.15.88:3300/0 conn(0x7fbce404fa90 0x7fbce404fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.838+0000 7fbcea800640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce4054270 0x7fbce40389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.839+0000 7fbce9e00640 1 -- 172.21.15.88:0/3687760836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce4054270 msgr2=0x7fbce40389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.839+0000 7fbce9e00640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce4054270 0x7fbce40389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.839+0000 7fbce9e00640 1 -- 172.21.15.88:0/3687760836 >> v1:172.21.15.132:6789/0 conn(0x7fbce40df420 legacy=0x7fbce40382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.839+0000 7fbce9e00640 1 -- 172.21.15.88:0/3687760836 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce4006dd0 con 0x7fbce404fa90 2024-04-26T19:59:52.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.841+0000 7fbce9e00640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce404fa90 0x7fbce404fe70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009a00 tx=0x7fbcdc02c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5ef8ac82b69247 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.842+0000 7fa40c000640 1 Processor -- start 2024-04-26T19:59:52.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.842+0000 7fbce8a00640 1 -- 172.21.15.88:0/3687760836 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc038070 con 0x7fbce404fa90 2024-04-26T19:59:52.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.842+0000 7fa40c000640 1 -- start start 2024-04-26T19:59:52.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.842+0000 7fbce8a00640 1 -- 172.21.15.88:0/3687760836 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc02ce50 con 0x7fbce404fa90 2024-04-26T19:59:52.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.842+0000 7fbce8a00640 1 -- 172.21.15.88:0/3687760836 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc033800 con 0x7fbce404fa90 2024-04-26T19:59:52.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.843+0000 7fbceb200640 1 -- 172.21.15.88:0/3687760836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce404fa90 msgr2=0x7fbce404fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.843+0000 7fbceb200640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce404fa90 0x7fbce404fe70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009a00 tx=0x7fbcdc02c630 comp rx=0 tx=0).stop 2024-04-26T19:59:52.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.844+0000 7fbceb200640 1 -- 172.21.15.88:0/3687760836 shutdown_connections 2024-04-26T19:59:52.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.844+0000 7fbceb200640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce4054270 0x7fbce40389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.844+0000 7fbceb200640 1 --2- 172.21.15.88:0/3687760836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce404fa90 0x7fbce404fe70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.844+0000 7fbceb200640 1 -- 172.21.15.88:0/3687760836 >> 172.21.15.88:0/3687760836 conn(0x7fbce4009fb0 msgr2=0x7fbce400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.845+0000 7fbceb200640 1 -- 172.21.15.88:0/3687760836 shutdown_connections 2024-04-26T19:59:52.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.845+0000 7fbceb200640 1 -- 172.21.15.88:0/3687760836 wait complete. 2024-04-26T19:59:52.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.845+0000 7fa409600640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa3fc0e30c0 0x7fa3fc057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52752/0 (socket says 172.21.15.88:52752) 2024-04-26T19:59:52.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.845+0000 7fa409600640 1 -- 172.21.15.88:0/3147289190 learned_addr learned my addr 172.21.15.88:0/3147289190 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.846+0000 7fa40c000640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.88:3300/0 conn(0x7fa3fc03e840 0x7fa3fc053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.846+0000 7fa40c000640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.132:3300/0 conn(0x7fa3fc054130 0x7fa3fc0068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.846+0000 7fa408c00640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.88:3300/0 conn(0x7fa3fc03e840 0x7fa3fc053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.846+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3fc058570 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.846+0000 7fa40a000640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.132:3300/0 conn(0x7fa3fc054130 0x7fa3fc0068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.847+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037003005 0 0) 0x7fa3fc058570 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.847+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa3fc0545a0 con 0x7fa3fc03e840 2024-04-26T19:59:52.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.847+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa3fc058850 con 0x7fa3fc054130 2024-04-26T19:59:52.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3ec003520 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fbceb200640 1 Processor -- start 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1567772663 0 0) 0x7fa3ec003520 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 >> v2:172.21.15.132:3300/0 conn(0x7fa3fc054130 msgr2=0x7fa3fc0068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.132:3300/0 conn(0x7fa3fc054130 0x7fa3fc0068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 >> v2:172.21.15.88:3300/0 conn(0x7fa3fc03e840 msgr2=0x7fa3fc053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.848+0000 7fa403e00640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.88:3300/0 conn(0x7fa3fc03e840 0x7fa3fc053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc058b30 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa3f0001670 con 0x7fa3fc0e30c0 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 >> v1:172.21.15.88:6789/0 conn(0x7fa3fc0e30c0 legacy=0x7fa3fc057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3ec0051c0 0x7fa3ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc0545a0 con 0x7fa3ec004800 2024-04-26T19:59:52.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc058850 con 0x7fa3ec0039b0 2024-04-26T19:59:52.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec00bf30 con 0x7fa3ec0051c0 2024-04-26T19:59:52.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa408c00640 1 --2- 172.21.15.88:0/3147289190 >> v2:172.21.15.88:3300/0 conn(0x7fa3fc03e840 0x7fa3fc053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.849+0000 7fa408c00640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fbceb200640 1 -- start start 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fa409600640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fa40a000640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3ec0051c0 0x7fa3ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fa409600640 1 -- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3ec0051c0 msgr2=0x7fa3ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fa409600640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3ec0051c0 0x7fa3ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.850+0000 7fa409600640 1 -- 172.21.15.88:0/3147289190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3ec004800 msgr2=0x7fa3ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fa409600640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fa409600640 1 -- 172.21.15.88:0/3147289190 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc058b30 con 0x7fa3ec0039b0 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fa408c00640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3ec004800 0x7fa3ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce404fa90 0x7fbce40d9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 0x7fbce40d8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7f23d7e00640 1 Processor -- start 2024-04-26T19:59:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce40df420 0x7fbce4035420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4054700 con 0x7fbce4054270 2024-04-26T19:59:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4007390 con 0x7fbce404fa90 2024-04-26T19:59:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.851+0000 7fbceb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbce4054880 con 0x7fbce40df420 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7fa40a000640 1 -- 172.21.15.88:0/3147289190 reap_dead start 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7f4db9000640 1 Processor -- start 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7fbce9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 0x7fbce40d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7fbce9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 0x7fbce40d8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44986/0 (socket says 172.21.15.88:44986) 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7fbce9400640 1 -- 172.21.15.88:0/328123581 learned_addr learned my addr 172.21.15.88:0/328123581 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7fbce9e00640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce404fa90 0x7fbce40d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7f4db9000640 1 -- start start 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbcea800640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce40df420 0x7fbce4035420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 -- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce40df420 msgr2=0x7fbce4035420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbce40df420 0x7fbce4035420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 -- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce404fa90 msgr2=0x7fbce40d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbce404fa90 0x7fbce40d9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce40d9960 con 0x7fbce4054270 2024-04-26T19:59:52.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.853+0000 7fbce9400640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 0x7fbce40d8340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800dfd0 tx=0x7fbcd800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7fa409600640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3ec003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c260 tx=0x7fa3ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9904bb5e2741ed52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8012c40 con 0x7fbce4054270 2024-04-26T19:59:52.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8016030 con 0x7fbce4054270 2024-04-26T19:59:52.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8012c40 con 0x7fbce4054270 2024-04-26T19:59:52.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce40d9c40 con 0x7fbce4054270 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f4db9000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4db414c880 0x7f4db41679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f4db9000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4073640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f4db9000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4db406f640 con 0x7f4db414cd50 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f4db9000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4db40d1570 con 0x7f4db414c880 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f4db9000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4db406fc00 con 0x7f4db40731f0 2024-04-26T19:59:52.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.852+0000 7f23d7e00640 1 -- start start 2024-04-26T19:59:52.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f48bc200640 1 Processor -- start 2024-04-26T19:59:52.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f48bc200640 1 -- start start 2024-04-26T19:59:52.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f23d815f270 0x7f23d80d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce40da200 con 0x7fbce4054270 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007d70 con 0x7fa3ec0039b0 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005af0 con 0x7fa3ec0039b0 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7fa403e00640 1 -- 172.21.15.88:0/3147289190 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007070 con 0x7fa3ec0039b0 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d1df0 0x7f23d80b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d80d0ed0 con 0x7f23d80d3c90 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f23d80d2540 con 0x7f23d815f270 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f23d80cfe30 con 0x7f23d80d1df0 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d6000640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f23d80d3c90 0x7f23d80d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48106/0 (socket says 172.21.15.88:48106) 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d6000640 1 -- 172.21.15.88:0/2617773438 learned_addr learned my addr 172.21.15.88:0/2617773438 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.854+0000 7f23d7400640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d1df0 0x7f23d80b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f23d6a00640 1 --2- 172.21.15.88:0/2617773438 >> v2:172.21.15.132:3300/0 conn(0x7f23d815f270 0x7f23d80d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f4daaa00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4db414cd50 0x7f4db406e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48114/0 (socket says 172.21.15.88:48114) 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f4daaa00640 1 -- 172.21.15.88:0/1243393543 learned_addr learned my addr 172.21.15.88:0/1243393543 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f4db3e00640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f23d5600640 1 -- 172.21.15.88:0/2617773438 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888478438 0 0) 0x7f23c4001110 con 0x7f23d80d3c90 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.855+0000 7f23d5600640 1 -- 172.21.15.88:0/2617773438 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23bc003520 con 0x7f23d80d3c90 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f23d6a00640 1 -- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d1df0 msgr2=0x7f23d80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f23d6a00640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d1df0 0x7f23d80b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f23d6a00640 1 -- 172.21.15.88:0/2617773438 >> v1:172.21.15.132:6789/0 conn(0x7f23d80d3c90 legacy=0x7f23d80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f23d6a00640 1 -- 172.21.15.88:0/2617773438 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d80d0110 con 0x7f23d815f270 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f48bc200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f48b40adf90 0x7f48b416d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f48bc200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 msgr2=0x7fa3ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7fa40c000640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3ec003dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c260 tx=0x7fa3ec005830 comp rx=0 tx=0).stop 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.856+0000 7f4db3400640 1 --2- 172.21.15.88:0/1243393543 >> v2:172.21.15.88:3300/0 conn(0x7f4db414c880 0x7f4db41679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbcc4001030 con 0x7fbce4054270 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 shutdown_connections 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7f4db3400640 1 -- 172.21.15.88:0/1243393543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 msgr2=0x7f4db4073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7f4db3400640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7f4db3400640 1 -- 172.21.15.88:0/1243393543 >> v1:172.21.15.132:6789/0 conn(0x7f4db414cd50 legacy=0x7f4db406e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7f4db3400640 1 -- 172.21.15.88:0/1243393543 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db406f920 con 0x7f4db414c880 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fa40c000640 1 --2- 172.21.15.88:0/3147289190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3ec003dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 >> 172.21.15.88:0/3147289190 conn(0x7fa3fc0559e0 msgr2=0x7fa3fc055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 shutdown_connections 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7f48b8e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f48b40ae370 0x7f48b4072ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48126/0 (socket says 172.21.15.88:48126) 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48bc200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48b406e220 con 0x7f48b40ae370 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48bc200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f48b406e500 con 0x7f48b40adf90 2024-04-26T19:59:52.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48bc200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f48b406e7e0 con 0x7f48b4073600 2024-04-26T19:59:52.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48b8e00640 1 -- 172.21.15.88:0/4269500739 learned_addr learned my addr 172.21.15.88:0/4269500739 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48b9800640 1 --2- 172.21.15.88:0/4269500739 >> v2:172.21.15.132:3300/0 conn(0x7f48b40adf90 0x7f48b416d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd801b3e0 con 0x7fbce4054270 2024-04-26T19:59:52.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.857+0000 7fa40c000640 1 -- 172.21.15.88:0/3147289190 wait complete. 2024-04-26T19:59:52.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f48ba200640 1 --2- 172.21.15.88:0/4269500739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.858+0000 7f23d6a00640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d815f270 0x7f23d80d3750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097a0 tx=0x7f23c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35eb99d126067af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f4db3400640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414c880 0x7f4db41679d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009d90 tx=0x7f4da402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b1bb71584ebf68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f48b3e00640 1 -- 172.21.15.88:0/4269500739 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 159320562 0 0) 0x7f48b406e220 con 0x7f48b40ae370 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f48b3e00640 1 -- 172.21.15.88:0/4269500739 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f489c003520 con 0x7f48b40ae370 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f48ba200640 1 -- 172.21.15.88:0/4269500739 >> v2:172.21.15.132:3300/0 conn(0x7f48b40adf90 msgr2=0x7f48b416d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f48ba200640 1 --2- 172.21.15.88:0/4269500739 >> v2:172.21.15.132:3300/0 conn(0x7f48b40adf90 0x7f48b416d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f48ba200640 1 -- 172.21.15.88:0/4269500739 >> v1:172.21.15.132:6789/0 conn(0x7f48b40ae370 legacy=0x7f48b4072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.859+0000 7f4db2a00640 1 -- 172.21.15.88:0/1243393543 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4007740 con 0x7f4db414c880 2024-04-26T19:59:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7f48ba200640 1 -- 172.21.15.88:0/4269500739 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b406eac0 con 0x7f48b4073600 2024-04-26T19:59:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7f23d5600640 1 -- 172.21.15.88:0/2617773438 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8037070 con 0x7f23d815f270 2024-04-26T19:59:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7f23d5600640 1 -- 172.21.15.88:0/2617773438 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8032440 con 0x7f23d815f270 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7f4db2a00640 1 -- 172.21.15.88:0/1243393543 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da40078e0 con 0x7f4db414c880 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7f4db2a00640 1 -- 172.21.15.88:0/1243393543 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4007cb0 con 0x7f4db414c880 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.860+0000 7fbce2000640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbccc06caf0 0x7fbccc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcd808fdd0 con 0x7fbce4054270 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fbce2000640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7fbccc074c90 0x7fbccc0770b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbccc077760 con 0x7fbccc074c90 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbcd80924d0 con 0x7fbce4054270 2024-04-26T19:59:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fbce9e00640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbccc06caf0 0x7fbccc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fa40c000640 1 Processor -- start 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 -- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d815f270 msgr2=0x7f23d80d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d5600640 1 -- 172.21.15.88:0/2617773438 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8032810 con 0x7f23d815f270 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d815f270 0x7f23d80d3750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097a0 tx=0x7f23c802a180 comp rx=0 tx=0).stop 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 -- 172.21.15.88:0/2617773438 shutdown_connections 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d1df0 0x7f23d80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 --2- 172.21.15.88:0/2617773438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d815f270 0x7f23d80d3750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f23d7e00640 1 -- 172.21.15.88:0/2617773438 >> 172.21.15.88:0/2617773438 conn(0x7f23d809e2e0 msgr2=0x7f23d80d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f48ba200640 1 --2- 172.21.15.88:0/4269500739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416eb60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48a80097a0 tx=0x7f48a802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce11c6e0a9fe7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f4db9000640 1 -- 172.21.15.88:0/1243393543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414c880 msgr2=0x7f4db41679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7f4db9000640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414c880 0x7f4db41679d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009d90 tx=0x7f4da402a540 comp rx=0 tx=0).stop 2024-04-26T19:59:52.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.861+0000 7fa40c000640 1 -- start start 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f23d7e00640 1 -- 172.21.15.88:0/2617773438 shutdown_connections 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f48b3e00640 1 -- 172.21.15.88:0/4269500739 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037070 con 0x7f48b4073600 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f48b3e00640 1 -- 172.21.15.88:0/4269500739 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8032440 con 0x7f48b4073600 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f48b3e00640 1 -- 172.21.15.88:0/4269500739 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8032810 con 0x7f48b4073600 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f23d7e00640 1 -- 172.21.15.88:0/2617773438 wait complete. 2024-04-26T19:59:52.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f3d43400640 1 Processor -- start 2024-04-26T19:59:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7fbcea800640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7fbccc074c90 0x7fbccc0770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f4db9000640 1 -- 172.21.15.88:0/1243393543 shutdown_connections 2024-04-26T19:59:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f4db9000640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f4db9000640 1 --2- 172.21.15.88:0/1243393543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414c880 0x7f4db41679d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f4db9000640 1 -- 172.21.15.88:0/1243393543 >> 172.21.15.88:0/1243393543 conn(0x7f4db40b27b0 msgr2=0x7f4db40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3fc04c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3fc051a00 0x7fa3fc17c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc17c920 0x7fa3fc0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa4aaa00640 1 Processor -- start 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fbce9e00640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbccc06caf0 0x7fbccc06efb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc035000 tx=0x7fbcdc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc0e34a0 con 0x7fa3fc17c920 2024-04-26T19:59:52.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc03bfc0 con 0x7fa3ec0039b0 2024-04-26T19:59:52.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40c000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc053bf0 con 0x7fa3fc051a00 2024-04-26T19:59:52.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.862+0000 7f3d43400640 1 -- start start 2024-04-26T19:59:52.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fbcea800640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7fbccc074c90 0x7fbccc0770b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa4aaa00640 1 -- start start 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7f4db9000640 1 -- 172.21.15.88:0/1243393543 shutdown_connections 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa40a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc17c920 0x7fa3fc0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa408c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3fc051a00 0x7fa3fc17c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa408c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3fc051a00 0x7fa3fc17c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43690/0 (socket says 172.21.15.88:43690) 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa408c00640 1 -- 172.21.15.88:0/4117291402 learned_addr learned my addr 172.21.15.88:0/4117291402 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7fa409600640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3fc04c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== osd.0 v2:172.21.15.88:6808/2647855198 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbccc077760 con 0x7fbccc074c90 2024-04-26T19:59:52.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7f48bc200640 1 -- 172.21.15.88:0/4269500739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 msgr2=0x7f48b416eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7f48bc200640 1 --2- 172.21.15.88:0/4269500739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416eb60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f48a80097a0 tx=0x7f48a802a180 comp rx=0 tx=0).stop 2024-04-26T19:59:52.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa40a000640 1 -- 172.21.15.88:0/4117291402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3fc051a00 msgr2=0x7fa3fc17c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa40a000640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa3fc051a00 0x7fa3fc17c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa40a000640 1 -- 172.21.15.88:0/4117291402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 msgr2=0x7fa3fc04c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa40a000640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3ec0039b0 0x7fa3fc04c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa40a000640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc051f30 con 0x7fa3fc17c920 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa4aaa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 0x7fa4a40d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.864+0000 7fa4aaa00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4aaa00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a40d3520 con 0x7fa4a40b4d00 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4aaa00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a40d40f0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4aaa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40d37c0 con 0x7fa4a4162c10 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f48bc200640 1 -- 172.21.15.88:0/4269500739 shutdown_connections 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f48bc200640 1 --2- 172.21.15.88:0/4269500739 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416eb60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f48bc200640 1 --2- 172.21.15.88:0/4269500739 >> v2:172.21.15.132:3300/0 conn(0x7f48b40adf90 0x7f48b416d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f48bc200640 1 -- 172.21.15.88:0/4269500739 >> 172.21.15.88:0/4269500739 conn(0x7f48b40b3240 msgr2=0x7f48b40d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3d3c06c770 0x7f3d3c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f23d7e00640 1 Processor -- start 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 0x7f3d3c163fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c0b5ff0 con 0x7f3d3c163b30 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3c06d110 con 0x7f3d3c06c770 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f3d43400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c164500 con 0x7f3d3c15dc80 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f23d7e00640 1 -- start start 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 0x7fa4a40d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4a3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4a3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45014/0 (socket says 172.21.15.88:45014) 2024-04-26T19:59:52.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7fa4a3400640 1 -- 172.21.15.88:0/3882134775 learned_addr learned my addr 172.21.15.88:0/3882134775 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.865+0000 7f48bc200640 1 -- 172.21.15.88:0/4269500739 shutdown_connections 2024-04-26T19:59:52.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43708/0 (socket says 172.21.15.88:43708) 2024-04-26T19:59:52.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 -- 172.21.15.88:0/495423163 learned_addr learned my addr 172.21.15.88:0/495423163 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7fa40a000640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc17c920 0x7fa3fc0c9a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc03bb50 tx=0x7fa3f8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d33e00640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.132:3300/0 conn(0x7f3d3c06c770 0x7f3d3c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d41400640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 0x7f3d3c163fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.863+0000 7f4db9000640 1 -- 172.21.15.88:0/1243393543 wait complete. 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f4db9000640 1 Processor -- start 2024-04-26T19:59:52.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f4db9000640 1 -- start start 2024-04-26T19:59:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 -- 172.21.15.88:0/495423163 >> v2:172.21.15.132:3300/0 conn(0x7f3d3c06c770 msgr2=0x7f3d3c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.132:3300/0 conn(0x7f3d3c06c770 0x7f3d3c06cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.866+0000 7f3d40a00640 1 -- 172.21.15.88:0/495423163 >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 msgr2=0x7f3d3c163fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543609366 0 0) 0x7fa4a40d3520 con 0x7fa4a40b4d00 2024-04-26T19:59:52.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f3d40a00640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 0x7f3d3c163fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f3d40a00640 1 -- 172.21.15.88:0/495423163 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c164740 con 0x7f3d3c15dc80 2024-04-26T19:59:52.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404050c30 con 0x7fa3fc17c920 2024-04-26T19:59:52.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa488003520 con 0x7fa4a40b4d00 2024-04-26T19:59:52.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a3e00640 1 -- 172.21.15.88:0/3882134775 >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 msgr2=0x7fa4a40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f3d41400640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 0x7f3d3c163fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d80d1df0 0x7f23d80ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f48bc200640 1 -- 172.21.15.88:0/4269500739 wait complete. 2024-04-26T19:59:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 <== mon.1 v1:172.21.15.132:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3378028762 0 0) 0x7fa488003520 con 0x7fa4a40b4d00 2024-04-26T19:59:52.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a3e00640 1 --2- 172.21.15.88:0/3882134775 >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f04e6000640 1 Processor -- start 2024-04-26T19:59:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d3c90 0x7f23d8205600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4165f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db414c880 0x7f4db4146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 0x7f23d8205d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80d09b0 con 0x7f23d815f270 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f23d80d06d0 con 0x7f23d80d1df0 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7f23d7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f23d80cfae0 con 0x7f23d80d3c90 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414cd50 0x7f4db4146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4073b80 con 0x7f4db414cd50 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4db406e2b0 con 0x7f4db414c880 2024-04-26T19:59:52.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db9000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4db40cfc30 con 0x7f4db40731f0 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f23d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 0x7f23d8205d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f8004030 con 0x7fa3fc17c920 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404063d80 con 0x7fa3fc17c920 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa40c000640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc052210 con 0x7fa3fc17c920 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7fa40c000640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc0527d0 con 0x7fa3fc17c920 2024-04-26T19:59:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a3e00640 1 -- 172.21.15.88:0/3882134775 >> v1:172.21.15.132:6789/0 conn(0x7fa4a40b4d00 legacy=0x7fa4a40b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.867+0000 7fa4a3e00640 1 -- 172.21.15.88:0/3882134775 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a40d3aa0 con 0x7fa4a4162c10 2024-04-26T19:59:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f23d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 0x7f23d8205d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45038/0 (socket says 172.21.15.88:45038) 2024-04-26T19:59:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f23d7400640 1 -- 172.21.15.88:0/1068451761 learned_addr learned my addr 172.21.15.88:0/1068451761 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f04e6000640 1 -- start start 2024-04-26T19:59:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f23d6a00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d80d1df0 0x7f23d80ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f23d6000640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d3c90 0x7f23d8205600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.868+0000 7f4db3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414cd50 0x7f4db4146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7fa4a3400640 1 --2- 172.21.15.88:0/3882134775 >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d6a00640 1 -- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d3c90 msgr2=0x7f23d8205600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d6a00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23d80d3c90 0x7f23d8205600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d6a00640 1 -- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 msgr2=0x7f23d8205d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f4db3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f4daaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db414c880 0x7f4db4146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f4db3400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4165f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43722/0 (socket says 172.21.15.88:43722) 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f4db3400640 1 -- 172.21.15.88:0/2925045307 learned_addr learned my addr 172.21.15.88:0/2925045307 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7fa40c000640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3fc0057e0 con 0x7fa3fc17c920 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d6a00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 0x7f23d8205d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d6a00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d8206290 con 0x7f23d80d1df0 2024-04-26T19:59:52.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.869+0000 7f23d7400640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23d815f270 0x7f23d8205d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7fa4a3e00640 1 --2- 172.21.15.88:0/3882134775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 0x7fa4a40d1a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edb140e069798ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f4db3400640 1 -- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db414c880 msgr2=0x7f4db4146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f4db3400640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4db414c880 0x7f4db4146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f4db3400640 1 -- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414cd50 msgr2=0x7f4db4146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f4db3400640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db414cd50 0x7f4db4146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f4db3400640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db41446d0 con 0x7f4db40731f0 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.870+0000 7f23d6a00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d80d1df0 0x7f23d80ccb60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23c8031000 tx=0x7f23c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494037070 con 0x7fa4a4162c10 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49402ab30 con 0x7fa4a4162c10 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7fa4a2a00640 1 -- 172.21.15.88:0/3882134775 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c0019f0 con 0x7fa4a4162c10 2024-04-26T19:59:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f4db3400640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4165f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009d90 tx=0x7f4da4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f3d40a00640 1 --2- 172.21.15.88:0/495423163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c06d310 tx=0x7f3d2c02a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9943bb8ff6bba06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8041030 con 0x7f23d80d1df0 2024-04-26T19:59:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c803d4e0 con 0x7f23d80d1df0 2024-04-26T19:59:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8037040 con 0x7f23d80d1df0 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4040070 con 0x7f4db40731f0 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da4031bc0 con 0x7f4db40731f0 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4040070 con 0x7f4db40731f0 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f4db9000640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db41449b0 con 0x7f4db40731f0 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 -- 172.21.15.88:0/3882134775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 msgr2=0x7fa4a40d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04df400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f04e015b160 0x7f04e00ae700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52764/0 (socket says 172.21.15.88:52764) 2024-04-26T19:59:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f48bc200640 1 Processor -- start 2024-04-26T19:59:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/3882134775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 0x7fa4a40d1a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402a0a0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04df400640 1 -- 172.21.15.88:0/3217391097 learned_addr learned my addr 172.21.15.88:0/3217391097 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:52.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04e6000640 1 --2- 172.21.15.88:0/3217391097 >> v2:172.21.15.88:3300/0 conn(0x7f04e00aee10 0x7f04e0072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 -- 172.21.15.88:0/3882134775 shutdown_connections 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/3882134775 >> v2:172.21.15.88:3300/0 conn(0x7fa4a40b3e70 0x7fa4a40d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/3882134775 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a4162c10 0x7fa4a40d1a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 -- 172.21.15.88:0/3882134775 >> 172.21.15.88:0/3882134775 conn(0x7fa4a409e3f0 msgr2=0x7fa4a40d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e009eec0 con 0x7f04e015b160 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e00730b0 con 0x7f04e0070580 2024-04-26T19:59:52.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f04e0073950 con 0x7f04e00aee10 2024-04-26T19:59:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f4db9000640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db4166470 con 0x7f4db40731f0 2024-04-26T19:59:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7f48bc200640 1 -- start start 2024-04-26T19:59:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 -- 172.21.15.88:0/3882134775 shutdown_connections 2024-04-26T19:59:52.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f3d3be00640 1 -- 172.21.15.88:0/495423163 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c037070 con 0x7f3d3c15dc80 2024-04-26T19:59:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f3d3be00640 1 -- 172.21.15.88:0/495423163 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c032440 con 0x7f3d3c15dc80 2024-04-26T19:59:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.871+0000 7f3d3be00640 1 -- 172.21.15.88:0/495423163 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c032810 con 0x7f3d3c15dc80 2024-04-26T19:59:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.872+0000 7fa4aaa00640 1 -- 172.21.15.88:0/3882134775 wait complete. 2024-04-26T19:59:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.873+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d8206560 con 0x7f23d80d1df0 2024-04-26T19:59:52.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.873+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d8206b20 con 0x7f23d80d1df0 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.873+0000 7f3d43400640 1 -- 172.21.15.88:0/495423163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 msgr2=0x7f3d3c167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.873+0000 7f3d43400640 1 --2- 172.21.15.88:0/495423163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c06d310 tx=0x7f3d2c02a140 comp rx=0 tx=0).stop 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 -- 172.21.15.88:0/495423163 shutdown_connections 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.88:3300/0 conn(0x7f3d3c163b30 0x7f3d3c163fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 --2- 172.21.15.88:0/495423163 >> v2:172.21.15.132:3300/0 conn(0x7f3d3c06c770 0x7f3d3c06cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 --2- 172.21.15.88:0/495423163 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c167c80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 -- 172.21.15.88:0/495423163 >> 172.21.15.88:0/495423163 conn(0x7f3d3c0b2d50 msgr2=0x7f3d3c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.873+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4db4144bf0 con 0x7f4db40731f0 2024-04-26T19:59:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 -- 172.21.15.88:0/495423163 shutdown_connections 2024-04-26T19:59:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3f8012040 con 0x7fa3fc17c920 2024-04-26T19:59:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596398712 0 0) 0x7f04e009eec0 con 0x7f04e015b160 2024-04-26T19:59:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04c4005730 con 0x7f04e015b160 2024-04-26T19:59:52.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> v2:172.21.15.88:3300/0 conn(0x7f04e00aee10 0x7f04e0072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23d80693c0 con 0x7f23d80d1df0 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b40adf90 0x7f48b41f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ae370 0x7f48b4201f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48b408cb20 con 0x7f48b40adf90 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f48b408c9a0 con 0x7f48b40ae370 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48bc200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f48b40b0150 con 0x7f48b4073600 2024-04-26T19:59:52.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079692007 0 0) 0x7f04e00730b0 con 0x7f04e0070580 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04c4005a90 con 0x7f04e0070580 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3658166015 0 0) 0x7f04c4005730 con 0x7f04e015b160 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 >> v2:172.21.15.88:3300/0 conn(0x7f04e00aee10 msgr2=0x7f04e0072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48b8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b40adf90 0x7f48b41f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48b8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b40adf90 0x7f48b41f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45060/0 (socket says 172.21.15.88:45060) 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 --2- 172.21.15.88:0/3217391097 >> v2:172.21.15.88:3300/0 conn(0x7f04e00aee10 0x7f04e0072a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 >> v1:172.21.15.132:6789/0 conn(0x7f04e0070580 legacy=0x7f04e00b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e0073390 con 0x7f04e015b160 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f04d0001500 con 0x7f04e015b160 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 >> v1:172.21.15.88:6789/0 conn(0x7f04e015b160 legacy=0x7f04e00ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04c4005f90 0x7f04c40063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 0x7f04c4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4007540 0x7f04c40147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0073950 con 0x7f04c4006a10 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4014d00 con 0x7f04c4007540 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f04c4014e80 con 0x7f04c4005f90 2024-04-26T19:59:52.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7fa401600640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa3e406c940 0x7fa3e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7fa4aaa00640 1 Processor -- start 2024-04-26T19:59:52.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48b8e00640 1 -- 172.21.15.88:0/3133575697 learned_addr learned my addr 172.21.15.88:0/3133575697 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.875+0000 7f48b9800640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7fa4aaa00640 1 -- start start 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f8057b70 con 0x7fa3fc17c920 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7fa409600640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa3e406c940 0x7fa3e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f48b9800640 1 -- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ae370 msgr2=0x7f48b4201f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f48b9800640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f48b40ae370 0x7f48b4201f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f48b9800640 1 -- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b40adf90 msgr2=0x7f48b41f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f48b9800640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b40adf90 0x7f48b41f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f48b9800640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b41f4830 con 0x7f48b4073600 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.876+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> v2:172.21.15.88:3300/0 conn(0x7f04e00aee10 0x7f04e0072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f04df400640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04c4005f90 0x7f04c40063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 0x7f04c4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f04dfe00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4007540 0x7f04c40147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7fa409600640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa3e406c940 0x7fa3e406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0033000 tx=0x7fa3f0007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7fa401600640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fa3e4075830 0x7fa3e4077cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3e40783a0 con 0x7fa3e4075830 2024-04-26T19:59:52.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa3f8056500 con 0x7fa3fc17c920 2024-04-26T19:59:52.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f48b9800640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416a5c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efc0 tx=0x7f48ac00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4da4039040 con 0x7f4db40731f0 2024-04-26T19:59:52.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac018020 con 0x7f48b4073600 2024-04-26T19:59:52.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f04d6a00640 1 -- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04c4005f90 msgr2=0x7f04c40063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04c4005f90 0x7f04c40063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac014480 con 0x7f48b4073600 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac014850 con 0x7f48b4073600 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 0x7fa4a40c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4a40b4d00 0x7fa4a40ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4a4162c10 0x7fa4a40caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40d0e40 con 0x7fa4a40b4d00 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40d0cc0 con 0x7fa4a4162c10 2024-04-26T19:59:52.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4aaa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40d0fc0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04d6a00640 1 -- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4007540 msgr2=0x7f04c40147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:52.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4007540 0x7f04c40147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04d6a00640 1 -- 172.21.15.88:0/3217391097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e0073390 con 0x7f04c4006a10 2024-04-26T19:59:52.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa408c00640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fa3e4075830 0x7fa3e4077cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 0x7fa4a40c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 0x7fa4a40c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43748/0 (socket says 172.21.15.88:43748) 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7fa4a3e00640 1 -- 172.21.15.88:0/163907206 learned_addr learned my addr 172.21.15.88:0/163907206 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04dfe00640 1 -- 172.21.15.88:0/3217391097 reap_dead start 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f04df400640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04c4005f90 0x7f04c40063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa408c00640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fa3e4075830 0x7fa3e4077cd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b41f4bb0 con 0x7f48b4073600 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41f5170 con 0x7f48b4073600 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f4dabe00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9006cb20 0x7f4d9006efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4da40ad2e0 con 0x7f4db40731f0 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f4dabe00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f4d90074cc0 0x7f4d900770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4da40020b0 con 0x7f4d90074cc0 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4da40ad6d0 con 0x7f4db40731f0 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f4daaa00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9006cb20 0x7f4d9006efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 -- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4a4162c10 msgr2=0x7fa4a40caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4a4162c10 0x7fa4a40caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 -- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4a40b4d00 msgr2=0x7fa4a40ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4a40b4d00 0x7fa4a40ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a40c8d20 con 0x7fa4a40b3e70 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7fa4a3e00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 0x7fa4a40c9aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098c0 tx=0x7fa4940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.874+0000 7f3d43400640 1 -- 172.21.15.88:0/495423163 wait complete. 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.877+0000 7f3d43400640 1 Processor -- start 2024-04-26T19:59:52.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.878+0000 7f3d43400640 1 -- start start 2024-04-26T19:59:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.879+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48b4069540 con 0x7f48b4073600 2024-04-26T19:59:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.880+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== osd.1 v2:172.21.15.88:6800/125701657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3e40783a0 con 0x7fa3e4075830 2024-04-26T19:59:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.880+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 0x7f04c4006e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006fd0 tx=0x7f04c4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e3403cb391dad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f4daaa00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9006cb20 0x7f4d9006efe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4db413c240 tx=0x7f4d9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.880+0000 7f3d43400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 0x7f3d3c144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.880+0000 7f3d43400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.880+0000 7f3d43400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d3c163b30 0x7f3d3c09c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8037070 con 0x7f04c4006a10 2024-04-26T19:59:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d40a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 0x7f3d3c144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c15e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43764/0 (socket says 172.21.15.88:43764) 2024-04-26T19:59:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c802aca0 con 0x7f04c4006a10 2024-04-26T19:59:52.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d40a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 0x7f3d3c144eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45078/0 (socket says 172.21.15.88:45078) 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f4db3e00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f4d90074cc0 0x7f4d900770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d43400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c167760 con 0x7f3d3c06c770 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d43400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c167480 con 0x7f3d3c163b30 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d43400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3c164a20 con 0x7f3d3c15dc80 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.881+0000 7f3d33e00640 1 -- 172.21.15.88:0/1886839857 learned_addr learned my addr 172.21.15.88:0/1886839857 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f04dea00640 1 -- 172.21.15.88:0/3217391097 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800f040 con 0x7f04c4006a10 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23c803d8b0 con 0x7f23d80d1df0 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494037050 con 0x7fa4a40b3e70 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa494043070 con 0x7fa4a40b3e70 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49403ca50 con 0x7fa4a40b3e70 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f3d41400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d3c163b30 0x7f3d3c09c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a40baac0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a40c92a0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 msgr2=0x7f04c4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f04e6000640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 0x7f04c4006e90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006fd0 tx=0x7f04c4007a40 comp rx=0 tx=0).stop 2024-04-26T19:59:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 shutdown_connections 2024-04-26T19:59:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f04e6000640 1 --2- 172.21.15.88:0/3217391097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c4006a10 0x7f04c4006e90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 >> 172.21.15.88:0/3217391097 conn(0x7f04e00b3240 msgr2=0x7f04e00d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f3d41400640 1 -- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 msgr2=0x7f3d3c15e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.882+0000 7f3d41400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d3c15dc80 0x7f3d3c15e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f3d41400640 1 -- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 msgr2=0x7f3d3c144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48ac005d10 con 0x7f48b4073600 2024-04-26T19:59:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f3d41400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 0x7f3d3c144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f3d41400640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c09d120 con 0x7f3d3c163b30 2024-04-26T19:59:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f3d40a00640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c06c770 0x7f3d3c144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f4db3e00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f4d90074cc0 0x7f4d900770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4a4069540 con 0x7fa4a40b3e70 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 shutdown_connections 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.883+0000 7f04e6000640 1 -- 172.21.15.88:0/3217391097 wait complete. 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.884+0000 7f3d41400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d3c163b30 0x7f3d3c09c740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400cc90 tx=0x7f3d3400a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.884+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400f680 con 0x7f3d3c163b30 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.884+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c09d400 con 0x7f3d3c163b30 2024-04-26T19:59:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c09d9c0 con 0x7f3d3c163b30 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d3400f840 con 0x7f3d3c163b30 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400fc30 con 0x7f3d3c163b30 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.884+0000 7f48b1600640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f489806c8f0 0x7f489806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48ac08d210 con 0x7f48b4073600 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f48b1600640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f4898074a90 0x7f4898076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4898077560 con 0x7f4898074a90 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f48ac08d6a0 con 0x7f48b4073600 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f48b8e00640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f489806c8f0 0x7f489806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23cea00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23b806c9c0 0x7f23b806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f23c80ac950 con 0x7f23d80d1df0 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23d6000640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23b806c9c0 0x7f23b806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23cea00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f23b8074be0 0x7f23b8077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23c80067f0 con 0x7f23b8074be0 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f23c80acd40 con 0x7f23d80d1df0 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.885+0000 7f04e6000640 1 Processor -- start 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.886+0000 7f04e6000640 1 -- start start 2024-04-26T19:59:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.886+0000 7f48ba200640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f4898074a90 0x7f4898076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.886+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d14001030 con 0x7f3d3c163b30 2024-04-26T19:59:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.886+0000 7f23d7400640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f23b8074be0 0x7f23b8077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.887+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== osd.4 v2:172.21.15.132:6816/861564477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4da40020b0 con 0x7f4d90074cc0 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.887+0000 7f48ba200640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f4898074a90 0x7f4898076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.887+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa49403cbf0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4006a10 0x7f04e006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04e006eec0 0x7f04e0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e01494c0 0x7f04e013f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04e015b540 con 0x7f04e006eec0 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f04e00b4670 con 0x7f04c4006a10 2024-04-26T19:59:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04e6000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f04e00d1350 con 0x7f04e01494c0 2024-04-26T19:59:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f23d6000640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23b806c9c0 0x7f23b806ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23d806db20 tx=0x7f23c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04df400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4006a10 0x7f04e006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04df400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4006a10 0x7f04e006fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43454/0 (socket says 172.21.15.88:43454) 2024-04-26T19:59:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04df400640 1 -- 172.21.15.88:0/3355999322 learned_addr learned my addr 172.21.15.88:0/3355999322 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.888+0000 7f04dfe00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e01494c0 0x7f04e013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f23d7400640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f23b8074be0 0x7f23b8077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04e006eec0 0x7f04e0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa49be00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa48406c940 0x7fa48406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4940acfe0 con 0x7fa4a40b3e70 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa4a3400640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa48406c940 0x7fa48406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa49be00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fa484074b60 0x7fa484076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa494001d30 con 0x7fa484074b60 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa4940ad390 con 0x7fa4a40b3e70 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f04dfe00640 1 -- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4006a10 msgr2=0x7f04e006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== osd.3 v2:172.21.15.88:6816/1302198393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4898077560 con 0x7f4898074a90 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f04dfe00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04c4006a10 0x7f04e006fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.889+0000 7f04dfe00640 1 -- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04e006eec0 msgr2=0x7f04e0156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.890+0000 7fa4a3400640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa48406c940 0x7fa48406ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa490009630 tx=0x7fa490006e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.890+0000 7fa4a8a00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fa484074b60 0x7fa484076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.890+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d3400e3e0 con 0x7f3d3c163b30 2024-04-26T19:59:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.890+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== osd.2 v2:172.21.15.88:6821/3051227728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23c80067f0 con 0x7f23b8074be0 2024-04-26T19:59:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f04dfe00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04e006eec0 0x7f04e0156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f04dfe00640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e009bcc0 con 0x7f04e01494c0 2024-04-26T19:59:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04e006eec0 0x7f04e0156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f3d39600640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1c06c8f0 0x7f3d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d3408c880 con 0x7f3d3c163b30 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f3d39600640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f3d1c074a90 0x7f3d1c076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d1c077560 con 0x7f3d1c074a90 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3d3408cc70 con 0x7f3d3c163b30 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f04dfe00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e01494c0 0x7f04e013f010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f04e00b39f0 tx=0x7f04d800ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.891+0000 7f48b8e00640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f489806c8f0 0x7f489806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48b408c3d0 tx=0x7f48a400d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.892+0000 7f3d33e00640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f3d1c074a90 0x7f3d1c076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.893+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8019070 con 0x7f04e01494c0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.893+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d8006e20 con 0x7f04e01494c0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.893+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8003b70 con 0x7f04e01494c0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.893+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e006e140 con 0x7f04e01494c0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.894+0000 7f3d33e00640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f3d1c074a90 0x7f3d1c076eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.893+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e006e6c0 con 0x7f04e01494c0 2024-04-26T19:59:52.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.894+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04e0069540 con 0x7f04e01494c0 2024-04-26T19:59:52.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.895+0000 7f3d40a00640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1c06c8f0 0x7f3d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.895+0000 7fa4a8a00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fa484074b60 0x7fa484076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.896+0000 7f3d40a00640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1c06c8f0 0x7f3d1c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c034000 tx=0x7f3d2c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.897+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== osd.7 v2:172.21.15.132:6808/3391944263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d1c077560 con 0x7f3d1c074a90 2024-04-26T19:59:52.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.899+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== osd.5 v2:172.21.15.132:6817/4023647186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa494001d30 con 0x7fa484074b60 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.899+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04d8003d10 con 0x7f04e01494c0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.901+0000 7f04d7e00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04bc06c9f0 0x7f04bc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.901+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04d808b630 con 0x7f04e01494c0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.901+0000 7f04d7e00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f04bc074c10 0x7f04bc077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.901+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04d8009e50 con 0x7f04bc074c10 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.901+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f04d808ba20 con 0x7f04e01494c0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.902+0000 7f04df400640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04bc06c9f0 0x7f04bc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.902+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f04bc074c10 0x7f04bc077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.903+0000 7f04df400640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04bc06c9f0 0x7f04bc06eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04e0157d20 tx=0x7f04d0007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.904+0000 7f04d6a00640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f04bc074c10 0x7f04bc077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.906+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== osd.6 v2:172.21.15.132:6800/2163437982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04d8009e50 con 0x7f04bc074c10 2024-04-26T19:59:52.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.906+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 --> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbcc4002dd0 con 0x7fbccc074c90 2024-04-26T19:59:52.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.907+0000 7fbce2000640 1 -- 172.21.15.88:0/328123581 <== osd.0 v2:172.21.15.88:6808/2647855198 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbcc4002dd0 con 0x7fbccc074c90 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.908+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7fbccc074c90 msgr2=0x7fbccc0770b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.908+0000 7fbceb200640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7fbccc074c90 0x7fbccc0770b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbccc06caf0 msgr2=0x7fbccc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbceb200640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbccc06caf0 0x7fbccc06efb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc035000 tx=0x7fbcdc008030 comp rx=0 tx=0).stop 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 msgr2=0x7fbce40d8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbceb200640 1 --2- 172.21.15.88:0/328123581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbce4054270 0x7fbce40d8340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800dfd0 tx=0x7fbcd800b950 comp rx=0 tx=0).stop 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbcea800640 1 -- 172.21.15.88:0/328123581 reap_dead start 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.909+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 shutdown_connections 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.910+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 >> 172.21.15.88:0/328123581 conn(0x7fbce4009fb0 msgr2=0x7fbce40529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.910+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 shutdown_connections 2024-04-26T19:59:52.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.910+0000 7fbceb200640 1 -- 172.21.15.88:0/328123581 wait complete. 2024-04-26T19:59:52.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.914+0000 7fa40c000640 1 -- 172.21.15.88:0/4117291402 --> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3fc178c10 con 0x7fa3e4075830 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.914+0000 7fa401600640 1 -- 172.21.15.88:0/4117291402 <== osd.1 v2:172.21.15.88:6800/125701657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3fc178c10 con 0x7fa3e4075830 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.915+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fa3e4075830 msgr2=0x7fa3e4077cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.915+0000 7fa3f6a00640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fa3e4075830 0x7fa3e4077cd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa3e406c940 msgr2=0x7fa3e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa3e406c940 0x7fa3e406ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0033000 tx=0x7fa3f0007000 comp rx=0 tx=0).stop 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc17c920 msgr2=0x7fa3fc0c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 --2- 172.21.15.88:0/4117291402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc17c920 0x7fa3fc0c9a20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc03bb50 tx=0x7fa3f8011cf0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa40a000640 1 -- 172.21.15.88:0/4117291402 reap_dead start 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 shutdown_connections 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.916+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 >> 172.21.15.88:0/4117291402 conn(0x7fa3fc0559e0 msgr2=0x7fa3fc007510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.917+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 shutdown_connections 2024-04-26T19:59:52.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.917+0000 7fa3f6a00640 1 -- 172.21.15.88:0/4117291402 wait complete. 2024-04-26T19:59:52.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.918+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 --> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23d8073860 con 0x7f23b8074be0 2024-04-26T19:59:52.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.919+0000 7f23cea00640 1 -- 172.21.15.88:0/1068451761 <== osd.2 v2:172.21.15.88:6821/3051227728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f23d8073860 con 0x7f23b8074be0 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.919+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f23b8074be0 msgr2=0x7f23b8077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.919+0000 7f23d7e00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f23b8074be0 0x7f23b8077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23b806c9c0 msgr2=0x7f23b806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23b806c9c0 0x7f23b806ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23d806db20 tx=0x7f23c4009040 comp rx=0 tx=0).stop 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d80d1df0 msgr2=0x7f23d80ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 --2- 172.21.15.88:0/1068451761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23d80d1df0 0x7f23d80ccb60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23c8031000 tx=0x7f23c80046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7400640 1 -- 172.21.15.88:0/1068451761 reap_dead start 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 shutdown_connections 2024-04-26T19:59:52.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.920+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 >> 172.21.15.88:0/1068451761 conn(0x7f23d809e2e0 msgr2=0x7f23d80d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.922+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 shutdown_connections 2024-04-26T19:59:52.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.922+0000 7f23d7e00640 1 -- 172.21.15.88:0/1068451761 wait complete. 2024-04-26T19:59:52.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.924+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 --> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4db4068980 con 0x7f4d90074cc0 2024-04-26T19:59:52.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.924+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 --> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f48b409ee40 con 0x7f4898074a90 2024-04-26T19:59:52.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.925+0000 7f48b1600640 1 -- 172.21.15.88:0/3133575697 <== osd.3 v2:172.21.15.88:6816/1302198393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f48b409ee40 con 0x7f4898074a90 2024-04-26T19:59:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.926+0000 7f4dabe00640 1 -- 172.21.15.88:0/2925045307 <== osd.4 v2:172.21.15.132:6816/861564477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4db4068980 con 0x7f4d90074cc0 2024-04-26T19:59:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.926+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f4898074a90 msgr2=0x7f4898076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.926+0000 7f48bc200640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f4898074a90 0x7f4898076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.926+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f489806c8f0 msgr2=0x7f489806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.926+0000 7f48bc200640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f489806c8f0 0x7f489806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48b408c3d0 tx=0x7f48a400d040 comp rx=0 tx=0).stop 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 msgr2=0x7f48b416a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f48bc200640 1 --2- 172.21.15.88:0/3133575697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f48b4073600 0x7f48b416a5c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00efc0 tx=0x7f48ac00c490 comp rx=0 tx=0).stop 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f48ba200640 1 -- 172.21.15.88:0/3133575697 reap_dead start 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 shutdown_connections 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 >> 172.21.15.88:0/3133575697 conn(0x7f48b40b3240 msgr2=0x7f48b40b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.928+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 shutdown_connections 2024-04-26T19:59:52.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.928+0000 7f48bc200640 1 -- 172.21.15.88:0/3133575697 wait complete. 2024-04-26T19:59:52.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.927+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 --> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d14002dd0 con 0x7f3d1c074a90 2024-04-26T19:59:52.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.928+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f4d90074cc0 msgr2=0x7f4d900770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.928+0000 7f3d39600640 1 -- 172.21.15.88:0/1886839857 <== osd.7 v2:172.21.15.132:6808/3391944263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3d14002dd0 con 0x7f3d1c074a90 2024-04-26T19:59:52.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.928+0000 7f4da8c00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f4d90074cc0 0x7f4d900770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9006cb20 msgr2=0x7f4d9006efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f4da8c00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9006cb20 0x7f4d9006efe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4db413c240 tx=0x7f4d9c009040 comp rx=0 tx=0).stop 2024-04-26T19:59:52.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 msgr2=0x7f4db4165f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f4da8c00640 1 --2- 172.21.15.88:0/2925045307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4db40731f0 0x7f4db4165f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009d90 tx=0x7f4da4004a40 comp rx=0 tx=0).stop 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f4db3e00640 1 -- 172.21.15.88:0/2925045307 reap_dead start 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f3d1c074a90 msgr2=0x7f3d1c076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f3d1c074a90 0x7f3d1c076eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1c06c8f0 msgr2=0x7f3d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1c06c8f0 0x7f3d1c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c034000 tx=0x7f3d2c02e000 comp rx=0 tx=0).stop 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d3c163b30 msgr2=0x7f3d3c09c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.929+0000 7f3d43400640 1 --2- 172.21.15.88:0/1886839857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d3c163b30 0x7f3d3c09c740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400cc90 tx=0x7f3d3400a650 comp rx=0 tx=0).stop 2024-04-26T19:59:52.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 shutdown_connections 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 >> 172.21.15.88:0/2925045307 conn(0x7f4db40b27b0 msgr2=0x7f4db40b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f3d41400640 1 -- 172.21.15.88:0/1886839857 reap_dead start 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 shutdown_connections 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 >> 172.21.15.88:0/1886839857 conn(0x7f3d3c0b2d50 msgr2=0x7f3d3c0adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 shutdown_connections 2024-04-26T19:59:52.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f3d43400640 1 -- 172.21.15.88:0/1886839857 wait complete. 2024-04-26T19:59:52.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 shutdown_connections 2024-04-26T19:59:52.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.930+0000 7f4da8c00640 1 -- 172.21.15.88:0/2925045307 wait complete. 2024-04-26T19:59:52.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.934+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 --> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4a40c96f0 con 0x7fa484074b60 2024-04-26T19:59:52.950 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:52.950 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T19:59:52.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.936+0000 7fa49be00640 1 -- 172.21.15.88:0/163907206 <== osd.5 v2:172.21.15.132:6817/4023647186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa4a40c96f0 con 0x7fa484074b60 2024-04-26T19:59:52.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.936+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fa484074b60 msgr2=0x7fa484076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.936+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fa484074b60 0x7fa484076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa48406c940 msgr2=0x7fa48406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa48406c940 0x7fa48406ee00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa490009630 tx=0x7fa490006e90 comp rx=0 tx=0).stop 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 msgr2=0x7fa4a40c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 --2- 172.21.15.88:0/163907206 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa4a40b3e70 0x7fa4a40c9aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa4940098c0 tx=0x7fa4940046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4a8a00640 1 -- 172.21.15.88:0/163907206 reap_dead start 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 shutdown_connections 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 >> 172.21.15.88:0/163907206 conn(0x7fa4a409e3f0 msgr2=0x7fa4a406dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 shutdown_connections 2024-04-26T19:59:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.937+0000 7fa4aaa00640 1 -- 172.21.15.88:0/163907206 wait complete. 2024-04-26T19:59:52.953 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-26T19:59:52.953 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T19:59:52.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.942+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 --> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04e00b1cd0 con 0x7f04bc074c10 2024-04-26T19:59:52.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.943+0000 7f04d7e00640 1 -- 172.21.15.88:0/3355999322 <== osd.6 v2:172.21.15.132:6800/2163437982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04e00b1cd0 con 0x7f04bc074c10 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.944+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f04bc074c10 msgr2=0x7f04bc077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.944+0000 7f04e6000640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f04bc074c10 0x7f04bc077030 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.944+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04bc06c9f0 msgr2=0x7f04bc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.944+0000 7f04e6000640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04bc06c9f0 0x7f04bc06eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04e0157d20 tx=0x7f04d0007040 comp rx=0 tx=0).stop 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e01494c0 msgr2=0x7f04e013f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 --2- 172.21.15.88:0/3355999322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04e01494c0 0x7f04e013f010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f04e00b39f0 tx=0x7f04d800ff70 comp rx=0 tx=0).stop 2024-04-26T19:59:52.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04dfe00640 1 -- 172.21.15.88:0/3355999322 reap_dead start 2024-04-26T19:59:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 shutdown_connections 2024-04-26T19:59:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 >> 172.21.15.88:0/3355999322 conn(0x7f04e00b3240 msgr2=0x7f04e00b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 shutdown_connections 2024-04-26T19:59:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:52.945+0000 7f04e6000640 1 -- 172.21.15.88:0/3355999322 wait complete. 2024-04-26T19:59:52.956 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:52.957 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T19:59:52.958 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:52.958 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T19:59:52.959 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:52.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T19:59:52.960 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-26T19:59:52.960 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T19:59:52.966 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:52.966 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T19:59:52.968 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:52.968 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T19:59:53.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.219+0000 7f522b200640 1 Processor -- start 2024-04-26T19:59:53.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.220+0000 7f4f69c00640 1 Processor -- start 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.221+0000 7f4f69c00640 1 -- start start 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.220+0000 7f522b200640 1 -- start start 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.221+0000 7f522b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522c15f270 0x7f522c0d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.221+0000 7f522b200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522c0d0ed0 con 0x7f522c0d3c90 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.221+0000 7f522b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522c0d2540 con 0x7f522c0d1df0 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.221+0000 7f522b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0cfe30 con 0x7f522c15f270 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.222+0000 7f5229400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f522c0d3c90 0x7f522c0d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52776/0 (socket says 172.21.15.88:52776) 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.222+0000 7f5229400640 1 -- 172.21.15.88:0/3430786667 learned_addr learned my addr 172.21.15.88:0/3430786667 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.222+0000 7f5229e00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522c15f270 0x7f522c0d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697710133 0 0) 0x7f521c001110 con 0x7f522c0d3c90 2024-04-26T19:59:53.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5210003520 con 0x7f522c0d3c90 2024-04-26T19:59:53.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2589996858 0 0) 0x7f5210003520 con 0x7f522c0d3c90 2024-04-26T19:59:53.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f4f69c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4f5c0e7810 0x7f4f5c038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522c15f270 msgr2=0x7f522c0d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f4f69c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4f5c0eb2f0 0x7f4f5c0eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f522c15f270 0x7f522c0d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 >> v1:172.21.15.132:6789/0 conn(0x7f522c0d1df0 legacy=0x7f522c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f4f69c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f5c0e7bf0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0d0110 con 0x7f522c0d3c90 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f4f69c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5c057f10 con 0x7f4f5c0e7810 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f4f69c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4f5c0581f0 con 0x7f4f5c0eb2f0 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.224+0000 7f4f62a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4f5c0e7810 0x7f4f5c038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.224+0000 7f4f62a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4f5c0e7810 0x7f4f5c038270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45090/0 (socket says 172.21.15.88:45090) 2024-04-26T19:59:53.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.224+0000 7f4f62a00640 1 -- 172.21.15.88:0/447021130 learned_addr learned my addr 172.21.15.88:0/447021130 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f521c001840 con 0x7f522c0d3c90 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 >> v1:172.21.15.88:6789/0 conn(0x7f522c0d3c90 legacy=0x7f522c0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52100039e0 0x7f5210003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210004800 0x7f5210004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210005210 0x7f521000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f522c0cfe30 con 0x7f5210005210 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f521000c140 con 0x7f52100039e0 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.223+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f521000c340 con 0x7f5210004800 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f4f62a00640 1 -- 172.21.15.88:0/447021130 >> v2:172.21.15.132:3300/0 conn(0x7f4f5c0eb2f0 msgr2=0x7f4f5c0eb770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f4f62a00640 1 --2- 172.21.15.88:0/447021130 >> v2:172.21.15.132:3300/0 conn(0x7f4f5c0eb2f0 0x7f4f5c0eb770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f4f62a00640 1 -- 172.21.15.88:0/447021130 >> v1:172.21.15.88:6789/0 conn(0x7f4f5c0ebcb0 legacy=0x7f4f5c053510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f4f62a00640 1 -- 172.21.15.88:0/447021130 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0584d0 con 0x7f4f5c0e7810 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f5229400640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210004800 0x7f5210004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.225+0000 7f5229e00640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52100039e0 0x7f5210003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.226+0000 7f522a800640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210005210 0x7f521000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.226+0000 7f4f62a00640 1 --2- 172.21.15.88:0/447021130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0e7810 0x7f4f5c038270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f64051c10 tx=0x7f4f640a6800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48019819a186074 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.226+0000 7f4f61600640 1 -- 172.21.15.88:0/447021130 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6406e8a0 con 0x7f4f5c0e7810 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.226+0000 7f4f61600640 1 -- 172.21.15.88:0/447021130 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6406ea80 con 0x7f4f5c0e7810 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.226+0000 7f4f61600640 1 -- 172.21.15.88:0/447021130 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f640ae3f0 con 0x7f4f5c0e7810 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.227+0000 7f522a800640 1 -- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210004800 msgr2=0x7f5210004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.227+0000 7f522a800640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210004800 0x7f5210004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.227+0000 7f522a800640 1 -- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52100039e0 msgr2=0x7f5210003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.227+0000 7f522a800640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52100039e0 0x7f5210003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.227+0000 7f522a800640 1 -- 172.21.15.88:0/3430786667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c0d0110 con 0x7f5210005210 2024-04-26T19:59:53.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.228+0000 7f4f69c00640 1 -- 172.21.15.88:0/447021130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0e7810 msgr2=0x7f4f5c038270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.228+0000 7f4f69c00640 1 --2- 172.21.15.88:0/447021130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0e7810 0x7f4f5c038270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f64051c10 tx=0x7f4f640a6800 comp rx=0 tx=0).stop 2024-04-26T19:59:53.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 -- 172.21.15.88:0/447021130 shutdown_connections 2024-04-26T19:59:53.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 --2- 172.21.15.88:0/447021130 >> v2:172.21.15.132:3300/0 conn(0x7f4f5c0eb2f0 0x7f4f5c0eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 --2- 172.21.15.88:0/447021130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0e7810 0x7f4f5c038270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 -- 172.21.15.88:0/447021130 >> 172.21.15.88:0/447021130 conn(0x7f4f5c0559d0 msgr2=0x7f4f5c055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 -- 172.21.15.88:0/447021130 shutdown_connections 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.229+0000 7f4f69c00640 1 -- 172.21.15.88:0/447021130 wait complete. 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.230+0000 7f522a800640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210005210 0x7f521000b9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52240044b0 tx=0x7f5210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2e1fd1759fd1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.230+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224037070 con 0x7f5210005210 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.230+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522402ad80 con 0x7f5210005210 2024-04-26T19:59:53.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.230+0000 7f5228a00640 1 -- 172.21.15.88:0/3430786667 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224004080 con 0x7f5210005210 2024-04-26T19:59:53.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7f7dd9e00640 1 Processor -- start 2024-04-26T19:59:53.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.230+0000 7fe64dc00640 1 Processor -- start 2024-04-26T19:59:53.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7fe64dc00640 1 -- start start 2024-04-26T19:59:53.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7f7dd9e00640 1 -- start start 2024-04-26T19:59:53.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7f522b200640 1 -- 172.21.15.88:0/3430786667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210005210 msgr2=0x7f521000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7f522b200640 1 --2- 172.21.15.88:0/3430786667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5210005210 0x7f521000b9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52240044b0 tx=0x7f5210005830 comp rx=0 tx=0).stop 2024-04-26T19:59:53.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7feb70e00640 1 Processor -- start 2024-04-26T19:59:53.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7fd22d000640 1 Processor -- start 2024-04-26T19:59:53.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7f522a800640 1 -- 172.21.15.88:0/3430786667 reap_dead start 2024-04-26T19:59:53.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7f522b200640 1 -- 172.21.15.88:0/3430786667 shutdown_connections 2024-04-26T19:59:53.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7f522b200640 1 -- 172.21.15.88:0/3430786667 >> 172.21.15.88:0/3430786667 conn(0x7f522c09e2e0 msgr2=0x7f522c0d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.231+0000 7feb70e00640 1 -- start start 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe6480b6380 0x7fe6480d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe6480c81f0 0x7fe6480d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe6480d11f0 con 0x7fe6480c81f0 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe6480b8200 con 0x7fe6480b6380 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fe64dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480d0c30 con 0x7fe6480d1c40 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7f522b200640 1 -- 172.21.15.88:0/3430786667 shutdown_connections 2024-04-26T19:59:53.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7f522b200640 1 -- 172.21.15.88:0/3430786667 wait complete. 2024-04-26T19:59:53.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fe646000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe6480c81f0 0x7fe6480d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fe646000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe6480c81f0 0x7fe6480d1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45112/0 (socket says 172.21.15.88:45112) 2024-04-26T19:59:53.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fe646000640 1 -- 172.21.15.88:0/1366151178 learned_addr learned my addr 172.21.15.88:0/1366151178 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40d2180 0x7f7dd40cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7f7dd9e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7dd40b5d10 0x7f7dd40d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fe646a00640 1 --2- 172.21.15.88:0/1366151178 >> v2:172.21.15.132:3300/0 conn(0x7fe6480b6380 0x7fe6480d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fe647400640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.232+0000 7fd22d000640 1 -- start start 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fd22d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 0x7fd22815f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fd22d000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 0x7fd2280d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fd22d000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2280b5d30 con 0x7fd22815ef60 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.233+0000 7fd22d000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd2280d3e40 con 0x7fd2280d34a0 2024-04-26T19:59:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fd22d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd2280b6010 con 0x7fd2280d5d90 2024-04-26T19:59:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fe646000640 1 -- 172.21.15.88:0/1366151178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 msgr2=0x7fe6480d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fe646000640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fe646000640 1 -- 172.21.15.88:0/1366151178 >> v2:172.21.15.132:3300/0 conn(0x7fe6480b6380 msgr2=0x7fe6480d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fe646000640 1 --2- 172.21.15.88:0/1366151178 >> v2:172.21.15.132:3300/0 conn(0x7fe6480b6380 0x7fe6480d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fe646000640 1 -- 172.21.15.88:0/1366151178 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6480d0f10 con 0x7fe6480c81f0 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd9e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd40d3520 con 0x7f7dd4162c10 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd3400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7dd40b5d10 0x7f7dd40d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd3400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7dd40b5d10 0x7f7dd40d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43486/0 (socket says 172.21.15.88:43486) 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd3400640 1 -- 172.21.15.88:0/1904744949 learned_addr learned my addr 172.21.15.88:0/1904744949 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd2a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f7dd4162c10 0x7f7dd40d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48168/0 (socket says 172.21.15.88:48168) 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7feb6c0d1350 0x7feb6c072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feb6c0ae700 0x7feb6c0aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c15b6a0 con 0x7feb6c0d1350 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c073950 con 0x7feb6c0ae700 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7feb70e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c09eec0 con 0x7feb6c070580 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7fd225400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 0x7fd22815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f7dd2000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40d2180 0x7f7dd40cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd40d40f0 con 0x7f7dd40b5d10 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd40d37c0 con 0x7f7dd40d2180 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7fd225e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd22815ef60 0x7fd2280b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48174/0 (socket says 172.21.15.88:48174) 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7fd225e00640 1 -- 172.21.15.88:0/3584315767 learned_addr learned my addr 172.21.15.88:0/3584315767 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f4f69c00640 1 Processor -- start 2024-04-26T19:59:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.234+0000 7f4f69c00640 1 -- start start 2024-04-26T19:59:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7feb69400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7feb6a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feb6c0ae700 0x7feb6c0aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7feb6a800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7feb6c0ae700 0x7feb6c0aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43490/0 (socket says 172.21.15.88:43490) 2024-04-26T19:59:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7feb6a800640 1 -- 172.21.15.88:0/3042363170 learned_addr learned my addr 172.21.15.88:0/3042363170 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7f7dd1600640 1 -- 172.21.15.88:0/1904744949 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978327381 0 0) 0x7f7dd40d3520 con 0x7f7dd4162c10 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 0x7f4f5c0e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f5c0eb2f0 0x7f4f5c0de780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 0x7f4f5c0df810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.235+0000 7fe646000640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480d1700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe64808f4b0 tx=0x7fe63802a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a358be4a21099422 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f7dd1600640 1 -- 172.21.15.88:0/1904744949 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db8003520 con 0x7f7dd4162c10 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c058cd0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0ec190 con 0x7f4f5c0eb2f0 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f4f69c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c0546e0 con 0x7f4f5c0e7810 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f7dd3400640 1 -- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40d2180 msgr2=0x7f7dd40cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f7dd3400640 1 --2- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40d2180 0x7f7dd40cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f7dd3400640 1 -- 172.21.15.88:0/1904744949 >> v1:172.21.15.132:6789/0 conn(0x7f7dd4162c10 legacy=0x7f7dd40d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f7dd3400640 1 -- 172.21.15.88:0/1904744949 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd40d3aa0 con 0x7f7dd40b5d10 2024-04-26T19:59:53.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7fd226800640 1 --2- 172.21.15.88:0/3584315767 >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 0x7fd2280d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7fd225400640 1 -- 172.21.15.88:0/3584315767 >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 msgr2=0x7fd2280d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7fe645600640 1 -- 172.21.15.88:0/1366151178 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638037070 con 0x7fe6480c81f0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7fe645600640 1 -- 172.21.15.88:0/1366151178 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63802ae10 con 0x7fe6480c81f0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7fe645600640 1 -- 172.21.15.88:0/1366151178 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638032660 con 0x7fe6480c81f0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 0x7f4f5c0df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 0x7f4f5c0e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f62a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 0x7f4f5c0e9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43810/0 (socket says 172.21.15.88:43810) 2024-04-26T19:59:53.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f62a00640 1 -- 172.21.15.88:0/822836971 learned_addr learned my addr 172.21.15.88:0/822836971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7fd224a00640 1 -- 172.21.15.88:0/3584315767 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495966879 0 0) 0x7fd2280b5d30 con 0x7fd22815ef60 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7f522b200640 1 Processor -- start 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f522b200640 1 -- start start 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7fd225400640 1 --2- 172.21.15.88:0/3584315767 >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 0x7fd2280d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7fd225400640 1 -- 172.21.15.88:0/3584315767 >> v1:172.21.15.132:6789/0 conn(0x7fd22815ef60 legacy=0x7fd2280b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7fd225400640 1 -- 172.21.15.88:0/3584315767 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2280b62f0 con 0x7fd2280d5d90 2024-04-26T19:59:53.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7fd226800640 1 --2- 172.21.15.88:0/3584315767 >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 0x7fd2280d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7feb6a800640 1 -- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 msgr2=0x7feb6c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 -- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 msgr2=0x7f4f5c0e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 0x7f4f5c0e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 -- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f5c0eb2f0 msgr2=0x7f4f5c0de780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f5c0eb2f0 0x7f4f5c0de780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.237+0000 7f4f63400640 1 -- 172.21.15.88:0/822836971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c0dfd50 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 -- 172.21.15.88:0/1366151178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 msgr2=0x7fe6480d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480d1700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe64808f4b0 tx=0x7fe63802a0c0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7feb6a800640 1 --2- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7feb6a800640 1 -- 172.21.15.88:0/3042363170 >> v2:172.21.15.88:3300/0 conn(0x7feb6c0d1350 msgr2=0x7feb6c072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7feb6a800640 1 --2- 172.21.15.88:0/3042363170 >> v2:172.21.15.88:3300/0 conn(0x7feb6c0d1350 0x7feb6c072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.236+0000 7feb6a800640 1 -- 172.21.15.88:0/3042363170 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0730b0 con 0x7feb6c0ae700 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 -- 172.21.15.88:0/1366151178 shutdown_connections 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1366151178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480d1700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1366151178 >> v2:172.21.15.132:3300/0 conn(0x7fe6480b6380 0x7fe6480d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 -- 172.21.15.88:0/1366151178 >> 172.21.15.88:0/1366151178 conn(0x7fe6480d4f10 msgr2=0x7fe6480d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f7dd3400640 1 --2- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd40b5d10 0x7f7dd40d01f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009e50 tx=0x7f7dcc02a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b2e7fa206fc54c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f4f63400640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 0x7f4f5c0df810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0ec0c0 tx=0x7f4f5000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7feb6a800640 1 --2- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c0aeb90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb6c070510 tx=0x7feb5c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf645c79ce588e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210005210 0x7f522c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 0x7f522c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 0x7f522c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f522c0d09b0 con 0x7f522c1544a0 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f522c0d06d0 con 0x7f522c153f50 2024-04-26T19:59:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7f522b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f522c0cfae0 con 0x7f5210005210 2024-04-26T19:59:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f7dd1600640 1 -- 172.21.15.88:0/1904744949 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc036070 con 0x7f7dd40b5d10 2024-04-26T19:59:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50015020 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7feb68a00640 1 -- 172.21.15.88:0/3042363170 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c037070 con 0x7feb6c0ae700 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7feb68a00640 1 -- 172.21.15.88:0/3042363170 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c032440 con 0x7feb6c0ae700 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7feb68a00640 1 -- 172.21.15.88:0/3042363170 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c032850 con 0x7feb6c0ae700 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50011830 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50015020 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f522a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 0x7f522c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f5229400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 0x7f522c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f5229400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 0x7f522c160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43498/0 (socket says 172.21.15.88:43498) 2024-04-26T19:59:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f5229400640 1 -- 172.21.15.88:0/740679892 learned_addr learned my addr 172.21.15.88:0/740679892 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c050130 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f5229e00640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210005210 0x7f522c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 -- 172.21.15.88:0/1366151178 shutdown_connections 2024-04-26T19:59:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.238+0000 7fe64dc00640 1 -- 172.21.15.88:0/1366151178 wait complete. 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f7dd1600640 1 -- 172.21.15.88:0/1904744949 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc031600 con 0x7f7dd40b5d10 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7f7dd1600640 1 -- 172.21.15.88:0/1904744949 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0319f0 con 0x7f7dd40b5d10 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd40b5d10 msgr2=0x7f7dd40d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd40b5d10 0x7f7dd40d01f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009e50 tx=0x7f7dcc02a6c0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 shutdown_connections 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.239+0000 7fd225400640 1 --2- 172.21.15.88:0/3584315767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 0x7fd22815f340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2140097a0 tx=0x7fd21402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f7d9654be58c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7feb70e00640 1 -- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 msgr2=0x7feb6c0aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7feb70e00640 1 --2- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c0aeb90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb6c070510 tx=0x7feb5c02a180 comp rx=0 tx=0).stop 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7fd224a00640 1 -- 172.21.15.88:0/3584315767 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214037070 con 0x7fd2280d5d90 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7fd224a00640 1 -- 172.21.15.88:0/3584315767 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd214032440 con 0x7fd2280d5d90 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7fd224a00640 1 -- 172.21.15.88:0/3584315767 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd214032810 con 0x7fd2280d5d90 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd40b5d10 0x7f7dd40d01f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/1904744949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40d2180 0x7f7dd40cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 >> 172.21.15.88:0/1904744949 conn(0x7f7dd409e3f0 msgr2=0x7f7dd40d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f522a800640 1 -- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210005210 msgr2=0x7f522c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f522a800640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210005210 0x7f522c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f522a800640 1 -- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 msgr2=0x7f522c160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f522a800640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 0x7f522c160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f522a800640 1 -- 172.21.15.88:0/740679892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c15f650 con 0x7f522c1544a0 2024-04-26T19:59:53.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.240+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c0506f0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.241+0000 7f522a800640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 0x7f522c1605a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5224031000 tx=0x7f5224003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.241+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 shutdown_connections 2024-04-26T19:59:53.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.241+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224043070 con 0x7f522c1544a0 2024-04-26T19:59:53.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5224006b50 con 0x7f522c1544a0 2024-04-26T19:59:53.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224037040 con 0x7f522c1544a0 2024-04-26T19:59:53.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.241+0000 7f7dd9e00640 1 -- 172.21.15.88:0/1904744949 wait complete. 2024-04-26T19:59:53.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7fd22d000640 1 -- 172.21.15.88:0/3584315767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 msgr2=0x7fd22815f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c15f930 con 0x7f522c1544a0 2024-04-26T19:59:53.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c15c5c0 con 0x7f522c1544a0 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7fe64dc00640 1 Processor -- start 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7feb70e00640 1 -- 172.21.15.88:0/3042363170 shutdown_connections 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7feb70e00640 1 --2- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c0aeb90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7feb70e00640 1 --2- 172.21.15.88:0/3042363170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7fe64dc00640 1 -- start start 2024-04-26T19:59:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7feb70e00640 1 --2- 172.21.15.88:0/3042363170 >> v2:172.21.15.88:3300/0 conn(0x7feb6c0d1350 0x7feb6c072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7feb70e00640 1 -- 172.21.15.88:0/3042363170 >> 172.21.15.88:0/3042363170 conn(0x7feb6c0b3240 msgr2=0x7feb6c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.242+0000 7fd22d000640 1 --2- 172.21.15.88:0/3584315767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 0x7fd22815f340 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd2140097a0 tx=0x7fd21402a210 comp rx=0 tx=0).stop 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.243+0000 7fd22d000640 1 -- 172.21.15.88:0/3584315767 shutdown_connections 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.243+0000 7fd22d000640 1 --2- 172.21.15.88:0/3584315767 >> v2:172.21.15.88:3300/0 conn(0x7fd2280d34a0 0x7fd2280d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.243+0000 7fd22d000640 1 --2- 172.21.15.88:0/3584315767 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd2280d5d90 0x7fd22815f340 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.243+0000 7fd22d000640 1 -- 172.21.15.88:0/3584315767 >> 172.21.15.88:0/3584315767 conn(0x7fd22809e3b0 msgr2=0x7fd2280d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c0693c0 con 0x7f522c1544a0 2024-04-26T19:59:53.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7fe64dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7fd22d000640 1 -- 172.21.15.88:0/3584315767 shutdown_connections 2024-04-26T19:59:53.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7fd22d000640 1 -- 172.21.15.88:0/3584315767 wait complete. 2024-04-26T19:59:53.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7fe64dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5c0057d0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.244+0000 7f7dd9e00640 1 Processor -- start 2024-04-26T19:59:53.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe646a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe64dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7f7dd9e00640 1 -- start start 2024-04-26T19:59:53.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe64dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6480d5be0 con 0x7fe6480c81f0 2024-04-26T19:59:53.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe64dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe6480b6760 con 0x7fe6480b6380 2024-04-26T19:59:53.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe64dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe6480d3230 con 0x7fe6480d1c40 2024-04-26T19:59:53.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe646000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe646000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45158/0 (socket says 172.21.15.88:45158) 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe647400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7fe647400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43830/0 (socket says 172.21.15.88:43830) 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7fe647400640 1 -- 172.21.15.88:0/1704752472 learned_addr learned my addr 172.21.15.88:0/1704752472 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f50018020 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5224046070 con 0x7f522c1544a0 2024-04-26T19:59:53.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7f5222000640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 0x7f520c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7f5229e00640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 0x7f520c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5224076f40 con 0x7f522c1544a0 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7feb70e00640 1 -- 172.21.15.88:0/3042363170 shutdown_connections 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7fe647400640 1 -- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 msgr2=0x7fe648158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.245+0000 7fe646a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43510/0 (socket says 172.21.15.88:43510) 2024-04-26T19:59:53.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 0x7f7dd414f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe647400640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe647400640 1 -- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 msgr2=0x7fe6480b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe647400640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe647400640 1 -- 172.21.15.88:0/1704752472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6480cc080 con 0x7fe6480d1c40 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe646000640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:53.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7fe646a00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:53.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.247+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd4162c10 0x7f7dd40ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45170/0 (socket says 172.21.15.88:45170) 2024-04-26T19:59:53.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7fe647400640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe640007e60 tx=0x7fe64000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 0x7f7dd414f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd2a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 0x7f7dd414f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43834/0 (socket says 172.21.15.88:43834) 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd2000640 1 -- 172.21.15.88:0/2957721700 learned_addr learned my addr 172.21.15.88:0/2957721700 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f5229e00640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 0x7f520c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d2720 tx=0x7f5218008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7fd22d000640 1 Processor -- start 2024-04-26T19:59:53.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.248+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40b3ff0 con 0x7f7dd40d2180 2024-04-26T19:59:53.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7fd22d000640 1 -- start start 2024-04-26T19:59:53.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40b3e70 con 0x7f7dd4162c10 2024-04-26T19:59:53.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.246+0000 7feb70e00640 1 -- 172.21.15.88:0/3042363170 wait complete. 2024-04-26T19:59:53.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7feb70e00640 1 Processor -- start 2024-04-26T19:59:53.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7f7dd3400640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd4162c10 0x7f7dd40ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd40b4170 con 0x7f7dd40b5d10 2024-04-26T19:59:53.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7feb70e00640 1 -- start start 2024-04-26T19:59:53.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640014420 con 0x7fe6480d1c40 2024-04-26T19:59:53.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6480cc360 con 0x7fe6480d1c40 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe640019070 con 0x7fe6480d1c40 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640014420 con 0x7fe6480d1c40 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6480cc920 con 0x7fe6480d1c40 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f7dd2000640 1 -- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 msgr2=0x7f7dd414f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7f4f5aa00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 0x7f4f4406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.249+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5008b3d0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f4f62a00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 0x7f4f4406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.251+0000 7feb70e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.251+0000 7fd22d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f7dd2000640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 0x7f7dd414f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f7dd2000640 1 -- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd4162c10 msgr2=0x7f7dd40ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f7dd2000640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd4162c10 0x7f7dd40ba6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.250+0000 7f7dd2000640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd414f9f0 con 0x7f7dd40d2180 2024-04-26T19:59:53.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb70e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd22d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 0x7fd2280c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd22d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd22815ef60 0x7fd2280b9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd22d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280d01d0 con 0x7fd2280d5d90 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd22d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd2280b68b0 con 0x7fd2280d34a0 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd22d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd2280d0350 con 0x7fd22815ef60 2024-04-26T19:59:53.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6480693c0 con 0x7fe6480d1c40 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb70e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 0x7feb6c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb70e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c0b4670 con 0x7feb6c0d1350 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb70e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c073670 con 0x7feb6c0ae700 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb70e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c15eda0 con 0x7feb6c070580 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7feb69e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7feb6a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 0x7feb6c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7feb6a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 0x7feb6c06f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45194/0 (socket says 172.21.15.88:45194) 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7feb69e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43846/0 (socket says 172.21.15.88:43846) 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7feb69e00640 1 -- 172.21.15.88:0/27735602 learned_addr learned my addr 172.21.15.88:0/27735602 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7feb69400640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd225e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd225e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:43532/0 (socket says 172.21.15.88:43532) 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd225e00640 1 -- 172.21.15.88:0/1944565302 learned_addr learned my addr 172.21.15.88:0/1944565302 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.252+0000 7fd225400640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 0x7fd2280c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7fd226800640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd22815ef60 0x7fd2280b9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7f4f62a00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 0x7f4f4406efe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f64051470 tx=0x7f4f640ac000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.253+0000 7f7dd2000640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e8a0 tx=0x7f7dc400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225e00640 1 -- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd22815ef60 msgr2=0x7fd2280b9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225e00640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd22815ef60 0x7fd2280b9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6400145c0 con 0x7fe6480d1c40 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225e00640 1 -- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 msgr2=0x7fd2280c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7feb69e00640 1 -- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 msgr2=0x7feb6c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7feb69e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c06f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7feb69e00640 1 -- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 msgr2=0x7feb6c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7feb69e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 0x7feb6c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7feb69e00640 1 -- 172.21.15.88:0/27735602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c09d230 con 0x7feb6c070580 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5224078ea0 con 0x7f522c1544a0 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225e00640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 0x7fd2280c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225e00640 1 -- 172.21.15.88:0/1944565302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2280c9e50 con 0x7fd2280d34a0 2024-04-26T19:59:53.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.254+0000 7fd225400640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 0x7fd2280c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.255+0000 7feb69e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb5800b410 tx=0x7feb5800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.255+0000 7fd225e00640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c8f0 tx=0x7fd21800cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.255+0000 7fe63ea00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 0x7fe62806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218018070 con 0x7fd2280d34a0 2024-04-26T19:59:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21800a640 con 0x7fd2280d34a0 2024-04-26T19:59:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21800aa30 con 0x7fd2280d34a0 2024-04-26T19:59:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2280ca130 con 0x7fd2280d34a0 2024-04-26T19:59:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe64008bfb0 con 0x7fe6480d1c40 2024-04-26T19:59:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.256+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22814edd0 con 0x7fd2280d34a0 2024-04-26T19:59:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.257+0000 7fe646a00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 0x7fe62806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.258+0000 7fe646a00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 0x7fe62806efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe634009a90 tx=0x7fe634008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.258+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228069540 con 0x7fd2280d34a0 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.258+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4017070 con 0x7f7dd40d2180 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.258+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd414fcd0 con 0x7f7dd40d2180 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.259+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd40bac20 con 0x7f7dd40d2180 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.259+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc400c940 con 0x7f7dd40d2180 2024-04-26T19:59:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.259+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4017070 con 0x7f7dd40d2180 2024-04-26T19:59:53.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.259+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f5001f020 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.260+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac005290 con 0x7f7dd40d2180 2024-04-26T19:59:53.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.261+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd21800abd0 con 0x7fd2280d34a0 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.262+0000 7fd21e000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 0x7fd20806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.262+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd21808b180 con 0x7fd2280d34a0 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.263+0000 7fd225400640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 0x7fd20806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.262+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dc401a070 con 0x7f7dd40d2180 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.264+0000 7f7dcb400640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 0x7f7db006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.264+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7dc408b890 con 0x7f7dd40d2180 2024-04-26T19:59:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.264+0000 7f7dd2a00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 0x7f7db006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.265+0000 7fd225400640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 0x7fd20806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd21402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.266+0000 7f7dd2a00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 0x7f7db006efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0011e0 tx=0x7f7dbc007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.268+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7dc4059660 con 0x7f7dd40d2180 2024-04-26T19:59:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58015020 con 0x7feb6c070580 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe64005a3d0 con 0x7fe6480d1c40 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb58011850 con 0x7feb6c070580 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58011c40 con 0x7feb6c070580 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c13f130 con 0x7feb6c070580 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.273+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c13ee50 con 0x7feb6c070580 2024-04-26T19:59:53.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.276+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c069540 con 0x7feb6c070580 2024-04-26T19:59:53.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.277+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd218058f40 con 0x7fd2280d34a0 2024-04-26T19:59:53.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.280+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb58018020 con 0x7feb6c070580 2024-04-26T19:59:53.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.284+0000 7feb62000640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 0x7feb4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.284+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb5808b940 con 0x7feb6c070580 2024-04-26T19:59:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.285+0000 7feb69400640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 0x7feb4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.287+0000 7feb69400640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 0x7feb4c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb6c06cb30 tx=0x7feb6400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.296+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb5801f020 con 0x7feb6c070580 2024-04-26T19:59:53.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.298+0000 7ffa03e00640 1 Processor -- start 2024-04-26T19:59:53.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.298+0000 7ffa03e00640 1 -- start start 2024-04-26T19:59:53.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.299+0000 7f2fdfc00640 1 Processor -- start 2024-04-26T19:59:53.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.300+0000 7ffa03e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ffa0415f1b0 0x7ffa040d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.299+0000 7f2fdfc00640 1 -- start start 2024-04-26T19:59:53.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.300+0000 7ffa03e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa040d3f50 con 0x7ffa04162b00 2024-04-26T19:59:53.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.300+0000 7ffa03e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa040d36b0 con 0x7ffa04163650 2024-04-26T19:59:53.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7ffa03e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ffa040d3990 con 0x7ffa0415f1b0 2024-04-26T19:59:53.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7f2fdfc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8072a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7f2fdfc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7ffa03400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7ffa04163650 0x7ffa040b82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:48178/0 (socket says 172.21.15.88:48178) 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7ffa03400640 1 -- 172.21.15.88:0/3219701481 learned_addr learned my addr 172.21.15.88:0/3219701481 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.301+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3219701481 >> v2:172.21.15.88:3300/0 conn(0x7ffa0415f1b0 0x7ffa040d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7ffa02a00640 1 -- 172.21.15.88:0/3219701481 >> v1:172.21.15.132:6789/0 conn(0x7ffa04163650 legacy=0x7ffa040b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7ffa02a00640 1 -- 172.21.15.88:0/3219701481 >> v1:172.21.15.88:6789/0 conn(0x7ffa04162b00 legacy=0x7ffa04162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7ffa02a00640 1 -- 172.21.15.88:0/3219701481 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa040d3c70 con 0x7ffa0415f1b0 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fdfc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd809eec0 con 0x7f2fd8070580 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fdfc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd8073950 con 0x7f2fd80d1350 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fdfc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80730b0 con 0x7f2fd815b160 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fcfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fcfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45220/0 (socket says 172.21.15.88:45220) 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fcfe00640 1 -- 172.21.15.88:0/2572806663 learned_addr learned my addr 172.21.15.88:0/2572806663 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.302+0000 7f2fdd200640 1 --2- 172.21.15.88:0/2572806663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.303+0000 7f2fd7e00640 1 -- 172.21.15.88:0/2572806663 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453131017 0 0) 0x7f2fd809eec0 con 0x7f2fd8070580 2024-04-26T19:59:53.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.303+0000 7f2fd7e00640 1 -- 172.21.15.88:0/2572806663 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb4003520 con 0x7f2fd8070580 2024-04-26T19:59:53.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.304+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3219701481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040d5990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4467c96ce2460d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.304+0000 7f2fdd200640 1 -- 172.21.15.88:0/2572806663 >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 msgr2=0x7f2fd815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:53.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.304+0000 7f2fdd200640 1 --2- 172.21.15.88:0/2572806663 >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.304+0000 7f2fdd200640 1 -- 172.21.15.88:0/2572806663 >> v1:172.21.15.132:6789/0 conn(0x7f2fd8070580 legacy=0x7f2fd815f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.304+0000 7f2fdd200640 1 -- 172.21.15.88:0/2572806663 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8073390 con 0x7f2fd815b160 2024-04-26T19:59:53.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.305+0000 7f2fcfe00640 1 --2- 172.21.15.88:0/2572806663 >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:59:53.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.306+0000 7ffa01600640 1 -- 172.21.15.88:0/3219701481 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc007740 con 0x7ffa0415f1b0 2024-04-26T19:59:53.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.306+0000 7ffa01600640 1 -- 172.21.15.88:0/3219701481 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0078e0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.306+0000 7ffa01600640 1 -- 172.21.15.88:0/3219701481 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc007cb0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.307+0000 7f2fdd200640 1 --2- 172.21.15.88:0/2572806663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8072a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40097a0 tx=0x7f2fc402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5bc16c0c80724fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.307+0000 7ffa03e00640 1 -- 172.21.15.88:0/3219701481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 msgr2=0x7ffa040d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.307+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3219701481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040d5990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc02a5b0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7f2fd7e00640 1 -- 172.21.15.88:0/2572806663 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4037070 con 0x7f2fd815b160 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7ffa03e00640 1 -- 172.21.15.88:0/3219701481 shutdown_connections 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3219701481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040d5990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7ffa03e00640 1 -- 172.21.15.88:0/3219701481 >> 172.21.15.88:0/3219701481 conn(0x7ffa0409e230 msgr2=0x7ffa040d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7f2fd7e00640 1 -- 172.21.15.88:0/2572806663 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc402aaf0 con 0x7f2fd815b160 2024-04-26T19:59:53.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7f2fd7e00640 1 -- 172.21.15.88:0/2572806663 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00019f0 con 0x7f2fd815b160 2024-04-26T19:59:53.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7ffa03e00640 1 -- 172.21.15.88:0/3219701481 shutdown_connections 2024-04-26T19:59:53.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.308+0000 7ffa03e00640 1 -- 172.21.15.88:0/3219701481 wait complete. 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 -- 172.21.15.88:0/2572806663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 msgr2=0x7f2fd8072a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/2572806663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8072a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40097a0 tx=0x7f2fc402a060 comp rx=0 tx=0).stop 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 -- 172.21.15.88:0/2572806663 shutdown_connections 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/2572806663 >> v2:172.21.15.88:3300/0 conn(0x7f2fd80d1350 0x7f2fd815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/2572806663 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8072a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.309+0000 7f2fdfc00640 1 -- 172.21.15.88:0/2572806663 >> 172.21.15.88:0/2572806663 conn(0x7f2fd80b3240 msgr2=0x7f2fd80d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.310+0000 7f2fdfc00640 1 -- 172.21.15.88:0/2572806663 shutdown_connections 2024-04-26T19:59:53.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.311+0000 7ffa03e00640 1 Processor -- start 2024-04-26T19:59:53.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.312+0000 7ffa03e00640 1 -- start start 2024-04-26T19:59:53.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.310+0000 7f2fdfc00640 1 -- 172.21.15.88:0/2572806663 wait complete. 2024-04-26T19:59:53.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.313+0000 7f2fdfc00640 1 Processor -- start 2024-04-26T19:59:53.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.313+0000 7f2fdfc00640 1 -- start start 2024-04-26T19:59:53.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.314+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.314+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 0x7ffa0415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa03e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa04163650 0x7ffa040ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa03e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040b89c0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa03e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040d33a0 con 0x7ffa04163650 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa03e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ffa040b8b40 con 0x7ffa04162b00 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7f2fdfc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa02000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 0x7ffa0415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa02000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 0x7ffa0415dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:43882/0 (socket says 172.21.15.88:43882) 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa02000640 1 -- 172.21.15.88:0/3936649993 learned_addr learned my addr 172.21.15.88:0/3936649993 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7ffa03400640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa04163650 0x7ffa040ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.315+0000 7f2fdfc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fd80d1350 0x7f2fd8150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fdfc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fdfc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd815b540 con 0x7f2fd8070580 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fdd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fdd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45248/0 (socket says 172.21.15.88:45248) 2024-04-26T19:59:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fdd200640 1 -- 172.21.15.88:0/819677838 learned_addr learned my addr 172.21.15.88:0/819677838 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7ffa02a00640 1 -- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 msgr2=0x7ffa0415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fcfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fd80d1350 0x7f2fd8150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7f2fddc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 0x7ffa0415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7ffa02a00640 1 -- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa04163650 msgr2=0x7ffa040ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa04163650 0x7ffa040ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.316+0000 7ffa02a00640 1 -- 172.21.15.88:0/3936649993 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa040cad60 con 0x7ffa0415f1b0 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.317+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80b4670 con 0x7f2fd80d1350 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.317+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd815f9f0 con 0x7f2fd815b160 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7f2fdd200640 1 -- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 msgr2=0x7f2fd8150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7f2fdd200640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7f2fdd200640 1 -- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fd80d1350 msgr2=0x7f2fd8150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7f2fdd200640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fd80d1350 0x7f2fd8150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7ffa02a00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040c9ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc031000 tx=0x7ff9fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7f2fdd200640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8142fc0 con 0x7f2fd8070580 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc040070 con 0x7ffa0415f1b0 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.318+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc031cb0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.319+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc031e60 con 0x7ffa0415f1b0 2024-04-26T19:59:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.319+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa040ca1e0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.319+0000 7f2fdd200640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4035040 tx=0x7f2fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.322+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4041070 con 0x7f2fd8070580 2024-04-26T19:59:53.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.322+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc403c660 con 0x7f2fd8070580 2024-04-26T19:59:53.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.322+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4041070 con 0x7f2fd8070580 2024-04-26T19:59:53.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.319+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa040ca7a0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.322+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd809c630 con 0x7f2fd8070580 2024-04-26T19:59:53.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.322+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd809cbf0 con 0x7f2fd8070580 2024-04-26T19:59:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.324+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd8069540 con 0x7f2fd8070580 2024-04-26T19:59:53.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.326+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9fc039040 con 0x7ffa0415f1b0 2024-04-26T19:59:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.326+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa041f50b0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.327+0000 7ff9fb400640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 0x7ff9e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.328+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff9fc0ace40 con 0x7ffa0415f1b0 2024-04-26T19:59:53.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.328+0000 7ffa02000640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 0x7ff9e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.330+0000 7ffa02000640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 0x7ff9e406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa040cec70 tx=0x7ff9f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.329+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fc4044070 con 0x7f2fd8070580 2024-04-26T19:59:53.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.331+0000 7f2fd5600640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 0x7f2fac06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:53.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.331+0000 7f2fcfe00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 0x7f2fac06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:53.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.333+0000 7f2fcfe00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 0x7f2fac06efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80d0080 tx=0x7f2fbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:53.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.333+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2fc40abed0 con 0x7f2fd8070580 2024-04-26T19:59:53.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.335+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fc4079db0 con 0x7f2fd8070580 2024-04-26T19:59:53.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.350+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa041f50b0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.678+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f522c15fe50 con 0x7f522c1544a0 2024-04-26T19:59:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.679+0000 7f5222000640 1 -- 172.21.15.88:0/740679892 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5224081a60 con 0x7f522c1544a0 2024-04-26T19:59:53.681 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-26T19:59:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.683+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 msgr2=0x7f520c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.683+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 0x7f520c06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d2720 tx=0x7f5218008040 comp rx=0 tx=0).stop 2024-04-26T19:59:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.683+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 msgr2=0x7f522c1605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.683+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 0x7f522c1605a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5224031000 tx=0x7f5224003a70 comp rx=0 tx=0).stop 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 shutdown_connections 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f522c1544a0 0x7f522c1605a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f522c153f50 0x7f522c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f520c06c8f0 0x7f520c06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 --2- 172.21.15.88:0/740679892 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5210005210 0x7f522c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.684+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 >> 172.21.15.88:0/740679892 conn(0x7f522c09e2e0 msgr2=0x7f522c06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.685+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 shutdown_connections 2024-04-26T19:59:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.685+0000 7f522b200640 1 -- 172.21.15.88:0/740679892 wait complete. 2024-04-26T19:59:53.700 INFO:teuthology.orchestra.run.smithi088.stdout:55834574853 2024-04-26T19:59:53.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.695+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7dac005580 con 0x7f7dd40d2180 2024-04-26T19:59:53.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.696+0000 7f7dcb400640 1 -- 172.21.15.88:0/2957721700 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dc405d960 con 0x7f7dd40d2180 2024-04-26T19:59:53.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.699+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 msgr2=0x7f7db006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.699+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 0x7f7db006efb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0011e0 tx=0x7f7dbc007d30 comp rx=0 tx=0).stop 2024-04-26T19:59:53.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.699+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 msgr2=0x7f7dd41f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.699+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400e8a0 tx=0x7f7dc400ed60 comp rx=0 tx=0).stop 2024-04-26T19:59:53.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 shutdown_connections 2024-04-26T19:59:53.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dd4162c10 0x7f7dd40ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dd40d2180 0x7f7dd41f8e70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7db006caf0 0x7f7db006efb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 --2- 172.21.15.88:0/2957721700 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7dd40b5d10 0x7f7dd414f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.700+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 >> 172.21.15.88:0/2957721700 conn(0x7f7dd409e3f0 msgr2=0x7f7dd406d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.701+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 shutdown_connections 2024-04-26T19:59:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.701+0000 7f7dd9e00640 1 -- 172.21.15.88:0/2957721700 wait complete. 2024-04-26T19:59:53.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.704+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4f5c0e70f0 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.704+0000 7f4f5aa00640 1 -- 172.21.15.88:0/822836971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f50059800 con 0x7f4f5c0ebcb0 2024-04-26T19:59:53.706 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:53.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.710+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 msgr2=0x7f4f4406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.710+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 0x7f4f4406efe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f64051470 tx=0x7f4f640ac000 comp rx=0 tx=0).stop 2024-04-26T19:59:53.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.710+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 msgr2=0x7f4f5c0df810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.710+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 0x7f4f5c0df810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0ec0c0 tx=0x7f4f5000b870 comp rx=0 tx=0).stop 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 shutdown_connections 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f5c0ebcb0 0x7f4f5c0df810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f5c0eb2f0 0x7f4f5c0de780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4f4406cb20 0x7f4f4406efe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 --2- 172.21.15.88:0/822836971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4f5c0e7810 0x7f4f5c0e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 >> 172.21.15.88:0/822836971 conn(0x7f4f5c0559d0 msgr2=0x7f4f5c005bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-26T19:59:53.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 shutdown_connections 2024-04-26T19:59:53.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.711+0000 7f4f69c00640 1 -- 172.21.15.88:0/822836971 wait complete. 2024-04-26T19:59:53.714 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.714+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd22808d090 con 0x7fd2280d34a0 2024-04-26T19:59:53.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.715+0000 7fd21e000640 1 -- 172.21.15.88:0/1944565302 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd21805d240 con 0x7fd2280d34a0 2024-04-26T19:59:53.717 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:53.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.716+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe6480d3a00 con 0x7fe6480d1c40 2024-04-26T19:59:53.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.717+0000 7fe63ea00640 1 -- 172.21.15.88:0/1704752472 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe640059cd0 con 0x7fe6480d1c40 2024-04-26T19:59:53.719 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:53.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 msgr2=0x7fd20806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 0x7fd20806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd214034000 tx=0x7fd21402e000 comp rx=0 tx=0).stop 2024-04-26T19:59:53.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 msgr2=0x7fd22814e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c8f0 tx=0x7fd21800cdb0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 shutdown_connections 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd20806c920 0x7fd20806ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd22815ef60 0x7fd2280b9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2280d5d90 0x7fd2280c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 --2- 172.21.15.88:0/1944565302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2280d34a0 0x7fd22814e5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.719+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 >> 172.21.15.88:0/1944565302 conn(0x7fd22809e3b0 msgr2=0x7fd2280d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.720+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 shutdown_connections 2024-04-26T19:59:53.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.720+0000 7fd22d000640 1 -- 172.21.15.88:0/1944565302 wait complete. 2024-04-26T19:59:53.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 msgr2=0x7fe62806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 0x7fe62806efb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe634009a90 tx=0x7fe634008040 comp rx=0 tx=0).stop 2024-04-26T19:59:53.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 msgr2=0x7fe6480b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe640007e60 tx=0x7fe64000e4b0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 shutdown_connections 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe62806caf0 0x7fe62806efb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6480d1c40 0x7fe6480b24f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6480c81f0 0x7fe6480b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 --2- 172.21.15.88:0/1704752472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6480b6380 0x7fe648158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 >> 172.21.15.88:0/1704752472 conn(0x7fe6480d4f10 msgr2=0x7fe6480ba020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 shutdown_connections 2024-04-26T19:59:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.723+0000 7fe64dc00640 1 -- 172.21.15.88:0/1704752472 wait complete. 2024-04-26T19:59:53.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-04-26T19:59:53.734 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-26T19:59:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.737+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7feb6c0741b0 con 0x7feb6c070580 2024-04-26T19:59:53.739 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.738+0000 7feb62000640 1 -- 172.21.15.88:0/27735602 <== mon.2 v2:172.21.15.132:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb6c09d230 con 0x7feb6c070580 2024-04-26T19:59:53.739 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 msgr2=0x7feb4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 0x7feb4c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb6c06cb30 tx=0x7feb6400b040 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 msgr2=0x7feb6c16bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb5800b410 tx=0x7feb5800b8d0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 shutdown_connections 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb6c0d1350 0x7feb6c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb6c0ae700 0x7feb6c06f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7feb4c06c8f0 0x7feb4c06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 --2- 172.21.15.88:0/27735602 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7feb6c070580 0x7feb6c16bd60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 >> 172.21.15.88:0/27735602 conn(0x7feb6c0b3240 msgr2=0x7feb6c0b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 shutdown_connections 2024-04-26T19:59:53.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.741+0000 7feb70e00640 1 -- 172.21.15.88:0/27735602 wait complete. 2024-04-26T19:59:53.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-26T19:59:53.746 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-26T19:59:53.747 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.750+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffa040693c0 con 0x7ffa0415f1b0 2024-04-26T19:59:53.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.751+0000 7ff9fb400640 1 -- 172.21.15.88:0/3936649993 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff9fc07ac00 con 0x7ffa0415f1b0 2024-04-26T19:59:53.752 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 msgr2=0x7ff9e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 0x7ff9e406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa040cec70 tx=0x7ff9f4009040 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 msgr2=0x7ffa040c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040c9ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc031000 tx=0x7ff9fc004a40 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 shutdown_connections 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa04163650 0x7ffa040ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff9e406c940 0x7ff9e406ee00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffa04162b00 0x7ffa0415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 --2- 172.21.15.88:0/3936649993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa0415f1b0 0x7ffa040c9ae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 >> 172.21.15.88:0/3936649993 conn(0x7ffa0409e230 msgr2=0x7ffa040d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 shutdown_connections 2024-04-26T19:59:53.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.753+0000 7ffa03e00640 1 -- 172.21.15.88:0/3936649993 wait complete. 2024-04-26T19:59:53.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.757+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2fd815a980 con 0x7f2fd8070580 2024-04-26T19:59:53.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.758+0000 7f2fd5600640 1 -- 172.21.15.88:0/819677838 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2fc407e0b0 con 0x7f2fd8070580 2024-04-26T19:59:53.759 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.760+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 msgr2=0x7f2fac06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.760+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 0x7f2fac06efb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80d0080 tx=0x7f2fbc008040 comp rx=0 tx=0).stop 2024-04-26T19:59:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.760+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 msgr2=0x7f2fd8155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.760+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4035040 tx=0x7f2fc40046c0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 shutdown_connections 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2fac06caf0 0x7f2fac06efb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2fd815b160 0x7f2fd8150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fd80d1350 0x7f2fd8150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 --2- 172.21.15.88:0/819677838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fd8070580 0x7f2fd8155a80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 >> 172.21.15.88:0/819677838 conn(0x7f2fd80b3240 msgr2=0x7f2fd80b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 shutdown_connections 2024-04-26T19:59:53.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:53.761+0000 7f2fdfc00640 1 -- 172.21.15.88:0/819677838 wait complete. 2024-04-26T19:59:53.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-26T19:59:53.765 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-04-26T19:59:53.771 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:53.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-26T19:59:54.780 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T19:59:54.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.992+0000 7f4b7c000640 1 Processor -- start 2024-04-26T19:59:54.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.992+0000 7f4b7c000640 1 -- start start 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b7c000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b7406c770 0x7f4b7406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b7c000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b7415cb70 0x7f4b74073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b7c000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b7406d090 con 0x7f4b7415cfd0 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b7c000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4b740af460 con 0x7f4b7406c770 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b7c000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4b740ae850 con 0x7f4b7415cb70 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b79600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b7406c770 0x7f4b7406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b79600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b7406c770 0x7f4b7406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45258/0 (socket says 172.21.15.88:45258) 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b79600640 1 -- 172.21.15.88:0/3504336253 learned_addr learned my addr 172.21.15.88:0/3504336253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:54.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b79600640 1 -- 172.21.15.88:0/3504336253 >> v2:172.21.15.132:3300/0 conn(0x7f4b7415cb70 msgr2=0x7f4b74073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.994+0000 7f4b79600640 1 --2- 172.21.15.88:0/3504336253 >> v2:172.21.15.132:3300/0 conn(0x7f4b7415cb70 0x7f4b74073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.995+0000 7f4b79600640 1 -- 172.21.15.88:0/3504336253 >> v1:172.21.15.132:6789/0 conn(0x7f4b7415cfd0 legacy=0x7f4b74073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.995+0000 7f4b79600640 1 -- 172.21.15.88:0/3504336253 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b740aeb30 con 0x7f4b7406c770 2024-04-26T19:59:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.996+0000 7f4b79600640 1 --2- 172.21.15.88:0/3504336253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7406c770 0x7f4b7406cb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b740afec0 tx=0x7f4b6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d6fbe6b50dc3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.996+0000 7f4b73e00640 1 -- 172.21.15.88:0/3504336253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64037070 con 0x7f4b7406c770 2024-04-26T19:59:54.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.996+0000 7f4b73e00640 1 -- 172.21.15.88:0/3504336253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64032440 con 0x7f4b7406c770 2024-04-26T19:59:54.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.996+0000 7f4b73e00640 1 -- 172.21.15.88:0/3504336253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64032810 con 0x7f4b7406c770 2024-04-26T19:59:54.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 -- 172.21.15.88:0/3504336253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7406c770 msgr2=0x7f4b7406cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3504336253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7406c770 0x7f4b7406cb50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b740afec0 tx=0x7f4b6402a230 comp rx=0 tx=0).stop 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 -- 172.21.15.88:0/3504336253 shutdown_connections 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3504336253 >> v2:172.21.15.132:3300/0 conn(0x7f4b7415cb70 0x7f4b74073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3504336253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7406c770 0x7f4b7406cb50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 -- 172.21.15.88:0/3504336253 >> 172.21.15.88:0/3504336253 conn(0x7f4b740b2d50 msgr2=0x7f4b740b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 -- 172.21.15.88:0/3504336253 shutdown_connections 2024-04-26T19:59:54.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.997+0000 7f4b7c000640 1 -- 172.21.15.88:0/3504336253 wait complete. 2024-04-26T19:59:55.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.999+0000 7f4b7c000640 1 Processor -- start 2024-04-26T19:59:55.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:54.999+0000 7f4b7c000640 1 -- start start 2024-04-26T19:59:55.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b7406c770 0x7f4b7409c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b7415cb70 0x7f4b7415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7415c600 con 0x7f4b7415cfd0 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4b740af0f0 con 0x7f4b7406c770 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7c000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7415c780 con 0x7f4b7415cb70 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45260/0 (socket says 172.21.15.88:45260) 2024-04-26T19:59:55.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 -- 172.21.15.88:0/3136478485 learned_addr learned my addr 172.21.15.88:0/3136478485 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b78c00640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b7415cb70 0x7f4b7415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b79600640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b7406c770 0x7f4b7409c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 -- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b7415cb70 msgr2=0x7f4b7415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b7415cb70 0x7f4b7415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 -- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b7406c770 msgr2=0x7f4b7409c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b7406c770 0x7f4b7409c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.000+0000 7f4b7a000640 1 -- 172.21.15.88:0/3136478485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7409e7d0 con 0x7f4b7415cfd0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b7a000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b970 tx=0x7f4b6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c019070 con 0x7f4b7415cfd0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b6c00cb40 con 0x7f4b7415cfd0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b74159b90 con 0x7f4b7415cfd0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c014440 con 0x7f4b7415cfd0 2024-04-26T19:59:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.001+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7415a150 con 0x7f4b7415cfd0 2024-04-26T19:59:55.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.002+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b74069540 con 0x7f4b7415cfd0 2024-04-26T19:59:55.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.003+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b6c01c070 con 0x7f4b7415cfd0 2024-04-26T19:59:55.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.004+0000 7f4b71600640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 0x7f4b5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:55.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.004+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b6c08b8b0 con 0x7f4b7415cfd0 2024-04-26T19:59:55.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.004+0000 7f4b79600640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 0x7f4b5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:55.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.005+0000 7f4b79600640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 0x7f4b5406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b64034000 tx=0x7f4b6402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:55.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.008+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b6c059bb0 con 0x7f4b7415cfd0 2024-04-26T19:59:55.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.194+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4b74154f90 con 0x7f4b7415cfd0 2024-04-26T19:59:55.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.195+0000 7f4b71600640 1 -- 172.21.15.88:0/3136478485 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b6c0594b0 con 0x7f4b7415cfd0 2024-04-26T19:59:55.197 INFO:teuthology.orchestra.run.smithi088.stdout:55834574854 2024-04-26T19:59:55.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 msgr2=0x7f4b5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 0x7f4b5406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b64034000 tx=0x7f4b6402e000 comp rx=0 tx=0).stop 2024-04-26T19:59:55.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 msgr2=0x7f4b7415e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:55.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b970 tx=0x7f4b6c00be30 comp rx=0 tx=0).stop 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 shutdown_connections 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b7415cfd0 0x7f4b7415e0c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b5406c8f0 0x7f4b5406edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b7415cb70 0x7f4b7415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 --2- 172.21.15.88:0/3136478485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b7406c770 0x7f4b7409c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.197+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 >> 172.21.15.88:0/3136478485 conn(0x7f4b740b2d50 msgr2=0x7f4b7406fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.198+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 shutdown_connections 2024-04-26T19:59:55.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:55.198+0000 7f4b7c000640 1 -- 172.21.15.88:0/3136478485 wait complete. 2024-04-26T19:59:55.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-04-26T19:59:56.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T19:59:56.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.424+0000 7f0fd3e00640 1 Processor -- start 2024-04-26T19:59:56.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.425+0000 7f0fd3e00640 1 -- start start 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0fd40d5990 0x7f0fd4163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0fd40b82b0 0x7f0fd40b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd40d3f50 con 0x7f0fd415f1b0 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd40d36b0 con 0x7f0fd40b82b0 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd40d3990 con 0x7f0fd40d5990 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0fd40b82b0 0x7f0fd40b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0fd40b82b0 0x7f0fd40b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45268/0 (socket says 172.21.15.88:45268) 2024-04-26T19:59:56.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.426+0000 7f0fd3400640 1 -- 172.21.15.88:0/2387507845 learned_addr learned my addr 172.21.15.88:0/2387507845 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:56.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.427+0000 7f0fd3400640 1 -- 172.21.15.88:0/2387507845 >> v2:172.21.15.132:3300/0 conn(0x7f0fd40d5990 msgr2=0x7f0fd4163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:56.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.427+0000 7f0fd3400640 1 --2- 172.21.15.88:0/2387507845 >> v2:172.21.15.132:3300/0 conn(0x7f0fd40d5990 0x7f0fd4163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.427+0000 7f0fd3400640 1 -- 172.21.15.88:0/2387507845 >> v1:172.21.15.132:6789/0 conn(0x7f0fd415f1b0 legacy=0x7f0fd4162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:56.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.427+0000 7f0fd3400640 1 -- 172.21.15.88:0/2387507845 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd40d3c70 con 0x7f0fd40b82b0 2024-04-26T19:59:56.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.428+0000 7f0fd3400640 1 --2- 172.21.15.88:0/2387507845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40b8740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009870 tx=0x7f0fcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43252e9d705b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.428+0000 7f0fd1600640 1 -- 172.21.15.88:0/2387507845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037070 con 0x7f0fd40b82b0 2024-04-26T19:59:56.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.428+0000 7f0fd1600640 1 -- 172.21.15.88:0/2387507845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc032440 con 0x7f0fd40b82b0 2024-04-26T19:59:56.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.428+0000 7f0fd1600640 1 -- 172.21.15.88:0/2387507845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc032810 con 0x7f0fd40b82b0 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 -- 172.21.15.88:0/2387507845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40b82b0 msgr2=0x7f0fd40b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/2387507845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40b8740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009870 tx=0x7f0fcc02a270 comp rx=0 tx=0).stop 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 -- 172.21.15.88:0/2387507845 shutdown_connections 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/2387507845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40b8740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/2387507845 >> v2:172.21.15.132:3300/0 conn(0x7f0fd40d5990 0x7f0fd4163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 -- 172.21.15.88:0/2387507845 >> 172.21.15.88:0/2387507845 conn(0x7f0fd409e230 msgr2=0x7f0fd40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 -- 172.21.15.88:0/2387507845 shutdown_connections 2024-04-26T19:59:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.429+0000 7f0fd3e00640 1 -- 172.21.15.88:0/2387507845 wait complete. 2024-04-26T19:59:56.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.431+0000 7f0fd3e00640 1 Processor -- start 2024-04-26T19:59:56.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.431+0000 7f0fd3e00640 1 -- start start 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd415f1b0 0x7f0fd406de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4163750 con 0x7f0fd40d5990 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd40d33a0 con 0x7f0fd40b82b0 2024-04-26T19:59:56.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd41638d0 con 0x7f0fd415f1b0 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45272/0 (socket says 172.21.15.88:45272) 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd2000640 1 -- 172.21.15.88:0/1902498584 learned_addr learned my addr 172.21.15.88:0/1902498584 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.432+0000 7f0fd2a00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 -- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd415f1b0 msgr2=0x7f0fd406de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd415f1b0 0x7f0fd406de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 -- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd40b82b0 msgr2=0x7f0fd40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 -- 172.21.15.88:0/1902498584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd415c050 con 0x7f0fd40d5990 2024-04-26T19:59:56.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd2000640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b4a0 tx=0x7f0fbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc014070 con 0x7f0fd40d5990 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc010610 con 0x7f0fd40d5990 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd40cde80 con 0x7f0fd40d5990 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc010a00 con 0x7f0fd40d5990 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.433+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd40ce440 con 0x7f0fd40d5990 2024-04-26T19:59:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.434+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd40693c0 con 0x7f0fd40d5990 2024-04-26T19:59:56.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.435+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fbc017070 con 0x7f0fd40d5990 2024-04-26T19:59:56.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.436+0000 7f0fcaa00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 0x7f0fb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.436+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0fbc08b640 con 0x7f0fd40d5990 2024-04-26T19:59:56.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.437+0000 7f0fd2a00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 0x7f0fb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.437+0000 7f0fd2a00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 0x7f0fb006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4007bf0 tx=0x7f0fc4007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.441+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fbc0599d0 con 0x7f0fd40d5990 2024-04-26T19:59:56.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.642+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0fd408d090 con 0x7f0fd40d5990 2024-04-26T19:59:56.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.642+0000 7f0fcaa00640 1 -- 172.21.15.88:0/1902498584 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0fbc0592d0 con 0x7f0fd40d5990 2024-04-26T19:59:56.644 INFO:teuthology.orchestra.run.smithi088.stdout:55834574855 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 msgr2=0x7f0fb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 0x7f0fb006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4007bf0 tx=0x7f0fc4007240 comp rx=0 tx=0).stop 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 msgr2=0x7f0fd40cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b4a0 tx=0x7f0fbc00b960 comp rx=0 tx=0).stop 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 shutdown_connections 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0fb006c8f0 0x7f0fb006edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0fd415f1b0 0x7f0fd406de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fd40d5990 0x7f0fd40cec40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 --2- 172.21.15.88:0/1902498584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fd40b82b0 0x7f0fd40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 >> 172.21.15.88:0/1902498584 conn(0x7f0fd409e230 msgr2=0x7f0fd40cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.645+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 shutdown_connections 2024-04-26T19:59:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.646+0000 7f0fd3e00640 1 -- 172.21.15.88:0/1902498584 wait complete. 2024-04-26T19:59:56.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-26T19:59:56.667 DEBUG:teuthology.parallel:result is None 2024-04-26T19:59:56.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T19:59:56.667 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:59:56.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.865+0000 7f8a67200640 1 Processor -- start 2024-04-26T19:59:56.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.866+0000 7f8a67200640 1 -- start start 2024-04-26T19:59:56.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a67200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a600d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a67200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a6015f800 con 0x7f8a6015f420 2024-04-26T19:59:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a67200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a600d4020 con 0x7f8a600c9520 2024-04-26T19:59:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a67200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a600d2c70 con 0x7f8a600d3680 2024-04-26T19:59:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a5fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8a6015f420 0x7f8a600d5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52792/0 (socket says 172.21.15.88:52792) 2024-04-26T19:59:56.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.867+0000 7f8a5fe00640 1 -- 172.21.15.88:0/2134802769 learned_addr learned my addr 172.21.15.88:0/2134802769 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5f400640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a600d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113904126 0 0) 0x7f8a6015f800 con 0x7f8a6015f420 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a44003520 con 0x7f8a6015f420 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133815297 0 0) 0x7f8a44003520 con 0x7f8a6015f420 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 msgr2=0x7f8a600d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a600d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 >> v1:172.21.15.132:6789/0 conn(0x7f8a600c9520 legacy=0x7f8a600c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a600d2ec0 con 0x7f8a6015f420 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f8a500019f0 con 0x7f8a6015f420 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 >> v1:172.21.15.88:6789/0 conn(0x7f8a6015f420 legacy=0x7f8a600d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a44003f60 0x7f8a44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44004d40 0x7f8a440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600d2c70 con 0x7f8a44003f60 2024-04-26T19:59:56.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4400bb50 con 0x7f8a44004d40 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4400bd50 con 0x7f8a600d3680 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.868+0000 7f8a5f400640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a44003f60 0x7f8a44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5f400640 1 -- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 msgr2=0x7f8a440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5fe00640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5f400640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600d3680 0x7f8a440039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5f400640 1 -- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44004d40 msgr2=0x7f8a440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5f400640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44004d40 0x7f8a440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.869+0000 7f8a5f400640 1 -- 172.21.15.88:0/2134802769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a44003520 con 0x7f8a44003f60 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.870+0000 7f8a5f400640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a44003f60 0x7f8a44004800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009ac0 tx=0x7f8a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749f7a4251ab1e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.870+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c036070 con 0x7f8a44003f60 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.870+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4c031400 con 0x7f8a44003f60 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.870+0000 7f8a5ea00640 1 -- 172.21.15.88:0/2134802769 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c009690 con 0x7f8a44003f60 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 -- 172.21.15.88:0/2134802769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a44003f60 msgr2=0x7f8a44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 --2- 172.21.15.88:0/2134802769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a44003f60 0x7f8a44004800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009ac0 tx=0x7f8a44005830 comp rx=0 tx=0).stop 2024-04-26T19:59:56.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a65200640 1 -- 172.21.15.88:0/2134802769 reap_dead start 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 -- 172.21.15.88:0/2134802769 shutdown_connections 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 -- 172.21.15.88:0/2134802769 >> 172.21.15.88:0/2134802769 conn(0x7f8a6009e3e0 msgr2=0x7f8a600d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 -- 172.21.15.88:0/2134802769 shutdown_connections 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.871+0000 7f8a67200640 1 -- 172.21.15.88:0/2134802769 wait complete. 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.873+0000 7f8a67200640 1 Processor -- start 2024-04-26T19:59:56.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.873+0000 7f8a67200640 1 -- start start 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44003f60 0x7f8a60160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600b8890 0x7f8a600b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600c8d20 con 0x7f8a600ccb80 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600d3480 con 0x7f8a44003f60 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a67200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a600c9000 con 0x7f8a600b8890 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45284/0 (socket says 172.21.15.88:45284) 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 -- 172.21.15.88:0/3583013519 learned_addr learned my addr 172.21.15.88:0/3583013519 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a65200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600b8890 0x7f8a600b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5fe00640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44003f60 0x7f8a60160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 -- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600b8890 msgr2=0x7f8a600b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600b8890 0x7f8a600b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 -- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44003f60 msgr2=0x7f8a60160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44003f60 0x7f8a60160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:56.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.874+0000 7f8a5f400640 1 -- 172.21.15.88:0/3583013519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a600cd640 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a5f400640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009be0 tx=0x7f8a4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c040020 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4c003bc0 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c036040 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a600b7df0 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.875+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a600b8090 con 0x7f8a600ccb80 2024-04-26T19:59:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.876+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a60069540 con 0x7f8a600ccb80 2024-04-26T19:59:56.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.877+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a4c043020 con 0x7f8a600ccb80 2024-04-26T19:59:56.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.878+0000 7f8a57e00640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:56.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.878+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a4c0abc30 con 0x7f8a600ccb80 2024-04-26T19:59:56.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.879+0000 7f8a5fe00640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:56.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.879+0000 7f8a5fe00640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a50001b60 tx=0x7f8a50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:56.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:56.883+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a4c04a070 con 0x7f8a600ccb80 2024-04-26T19:59:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.080+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a600d15b0 con 0x7f8a4006c8f0 2024-04-26T19:59:57.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.091+0000 7f8a57e00640 1 -- 172.21.15.88:0/3583013519 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f8a600d15b0 con 0x7f8a4006c8f0 2024-04-26T19:59:57.095 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T19:59:57.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 msgr2=0x7f8a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 0x7f8a4006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a50001b60 tx=0x7f8a50009040 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 msgr2=0x7f8a60153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009be0 tx=0x7f8a4c004730 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 shutdown_connections 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4006c8f0 0x7f8a4006edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a600b8890 0x7f8a600b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a600ccb80 0x7f8a60153380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 --2- 172.21.15.88:0/3583013519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a44003f60 0x7f8a60160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 >> 172.21.15.88:0/3583013519 conn(0x7f8a6009e3e0 msgr2=0x7f8a600c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 shutdown_connections 2024-04-26T19:59:57.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.097+0000 7f8a67200640 1 -- 172.21.15.88:0/3583013519 wait complete. 2024-04-26T19:59:57.117 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T19:59:56.943938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003330"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465394+0000","last_change":"2024-04-26T19:59:46.299176+0000","last_active":"2024-04-26T19:59:47.465394+0000","last_peered":"2024-04-26T19:59:47.465394+0000","last_clean":"2024-04-26T19:59:47.465394+0000","last_became_active":"2024-04-26T19:59:46.298473+0000","last_became_peered":"2024-04-26T19:59:46.298473+0000","last_unstale":"2024-04-26T19:59:47.465394+0000","last_undegraded":"2024-04-26T19:59:47.465394+0000","last_fullsized":"2024-04-26T19:59:47.465394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294427+0000","last_change":"2024-04-26T19:59:46.279518+0000","last_active":"2024-04-26T19:59:47.294427+0000","last_peered":"2024-04-26T19:59:47.294427+0000","last_clean":"2024-04-26T19:59:47.294427+0000","last_became_active":"2024-04-26T19:59:46.279235+0000","last_became_peered":"2024-04-26T19:59:46.279235+0000","last_unstale":"2024-04-26T19:59:47.294427+0000","last_undegraded":"2024-04-26T19:59:47.294427+0000","last_fullsized":"2024-04-26T19:59:47.294427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465545+0000","last_change":"2024-04-26T19:59:46.293998+0000","last_active":"2024-04-26T19:59:47.465545+0000","last_peered":"2024-04-26T19:59:47.465545+0000","last_clean":"2024-04-26T19:59:47.465545+0000","last_became_active":"2024-04-26T19:59:46.293698+0000","last_became_peered":"2024-04-26T19:59:46.293698+0000","last_unstale":"2024-04-26T19:59:47.465545+0000","last_undegraded":"2024-04-26T19:59:47.465545+0000","last_fullsized":"2024-04-26T19:59:47.465545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465774+0000","last_change":"2024-04-26T19:59:46.295413+0000","last_active":"2024-04-26T19:59:47.465774+0000","last_peered":"2024-04-26T19:59:47.465774+0000","last_clean":"2024-04-26T19:59:47.465774+0000","last_became_active":"2024-04-26T19:59:46.294973+0000","last_became_peered":"2024-04-26T19:59:46.294973+0000","last_unstale":"2024-04-26T19:59:47.465774+0000","last_undegraded":"2024-04-26T19:59:47.465774+0000","last_fullsized":"2024-04-26T19:59:47.465774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469290+0000","last_change":"2024-04-26T19:59:46.299569+0000","last_active":"2024-04-26T19:59:47.469290+0000","last_peered":"2024-04-26T19:59:47.469290+0000","last_clean":"2024-04-26T19:59:47.469290+0000","last_became_active":"2024-04-26T19:59:46.299229+0000","last_became_peered":"2024-04-26T19:59:46.299229+0000","last_unstale":"2024-04-26T19:59:47.469290+0000","last_undegraded":"2024-04-26T19:59:47.469290+0000","last_fullsized":"2024-04-26T19:59:47.469290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464401+0000","last_change":"2024-04-26T19:59:46.287705+0000","last_active":"2024-04-26T19:59:47.464401+0000","last_peered":"2024-04-26T19:59:47.464401+0000","last_clean":"2024-04-26T19:59:47.464401+0000","last_became_active":"2024-04-26T19:59:46.287398+0000","last_became_peered":"2024-04-26T19:59:46.287398+0000","last_unstale":"2024-04-26T19:59:47.464401+0000","last_undegraded":"2024-04-26T19:59:47.464401+0000","last_fullsized":"2024-04-26T19:59:47.464401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466440+0000","last_change":"2024-04-26T19:59:46.304005+0000","last_active":"2024-04-26T19:59:47.466440+0000","last_peered":"2024-04-26T19:59:47.466440+0000","last_clean":"2024-04-26T19:59:47.466440+0000","last_became_active":"2024-04-26T19:59:46.303696+0000","last_became_peered":"2024-04-26T19:59:46.303696+0000","last_unstale":"2024-04-26T19:59:47.466440+0000","last_undegraded":"2024-04-26T19:59:47.466440+0000","last_fullsized":"2024-04-26T19:59:47.466440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465106+0000","last_change":"2024-04-26T19:59:46.294993+0000","last_active":"2024-04-26T19:59:47.465106+0000","last_peered":"2024-04-26T19:59:47.465106+0000","last_clean":"2024-04-26T19:59:47.465106+0000","last_became_active":"2024-04-26T19:59:46.294405+0000","last_became_peered":"2024-04-26T19:59:46.294405+0000","last_unstale":"2024-04-26T19:59:47.465106+0000","last_undegraded":"2024-04-26T19:59:47.465106+0000","last_fullsized":"2024-04-26T19:59:47.465106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465470+0000","last_change":"2024-04-26T19:59:46.311322+0000","last_active":"2024-04-26T19:59:47.465470+0000","last_peered":"2024-04-26T19:59:47.465470+0000","last_clean":"2024-04-26T19:59:47.465470+0000","last_became_active":"2024-04-26T19:59:46.308499+0000","last_became_peered":"2024-04-26T19:59:46.308499+0000","last_unstale":"2024-04-26T19:59:47.465470+0000","last_undegraded":"2024-04-26T19:59:47.465470+0000","last_fullsized":"2024-04-26T19:59:47.465470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469431+0000","last_change":"2024-04-26T19:59:46.292325+0000","last_active":"2024-04-26T19:59:47.469431+0000","last_peered":"2024-04-26T19:59:47.469431+0000","last_clean":"2024-04-26T19:59:47.469431+0000","last_became_active":"2024-04-26T19:59:46.288719+0000","last_became_peered":"2024-04-26T19:59:46.288719+0000","last_unstale":"2024-04-26T19:59:47.469431+0000","last_undegraded":"2024-04-26T19:59:47.469431+0000","last_fullsized":"2024-04-26T19:59:47.469431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465975+0000","last_change":"2024-04-26T19:59:46.290820+0000","last_active":"2024-04-26T19:59:47.465975+0000","last_peered":"2024-04-26T19:59:47.465975+0000","last_clean":"2024-04-26T19:59:47.465975+0000","last_became_active":"2024-04-26T19:59:46.289935+0000","last_became_peered":"2024-04-26T19:59:46.289935+0000","last_unstale":"2024-04-26T19:59:47.465975+0000","last_undegraded":"2024-04-26T19:59:47.465975+0000","last_fullsized":"2024-04-26T19:59:47.465975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298228+0000","last_change":"2024-04-26T19:59:46.307273+0000","last_active":"2024-04-26T19:59:47.298228+0000","last_peered":"2024-04-26T19:59:47.298228+0000","last_clean":"2024-04-26T19:59:47.298228+0000","last_became_active":"2024-04-26T19:59:46.306426+0000","last_became_peered":"2024-04-26T19:59:46.306426+0000","last_unstale":"2024-04-26T19:59:47.298228+0000","last_undegraded":"2024-04-26T19:59:47.298228+0000","last_fullsized":"2024-04-26T19:59:47.298228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:35.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463934+0000","last_change":"2024-04-26T19:59:46.304263+0000","last_active":"2024-04-26T19:59:47.463934+0000","last_peered":"2024-04-26T19:59:47.463934+0000","last_clean":"2024-04-26T19:59:47.463934+0000","last_became_active":"2024-04-26T19:59:46.303977+0000","last_became_peered":"2024-04-26T19:59:46.303977+0000","last_unstale":"2024-04-26T19:59:47.463934+0000","last_undegraded":"2024-04-26T19:59:47.463934+0000","last_fullsized":"2024-04-26T19:59:47.463934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464499+0000","last_change":"2024-04-26T19:59:46.280420+0000","last_active":"2024-04-26T19:59:47.464499+0000","last_peered":"2024-04-26T19:59:47.464499+0000","last_clean":"2024-04-26T19:59:47.464499+0000","last_became_active":"2024-04-26T19:59:46.279398+0000","last_became_peered":"2024-04-26T19:59:46.279398+0000","last_unstale":"2024-04-26T19:59:47.464499+0000","last_undegraded":"2024-04-26T19:59:47.464499+0000","last_fullsized":"2024-04-26T19:59:47.464499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293703+0000","last_change":"2024-04-26T19:59:46.310086+0000","last_active":"2024-04-26T19:59:47.293703+0000","last_peered":"2024-04-26T19:59:47.293703+0000","last_clean":"2024-04-26T19:59:47.293703+0000","last_became_active":"2024-04-26T19:59:46.309760+0000","last_became_peered":"2024-04-26T19:59:46.309760+0000","last_unstale":"2024-04-26T19:59:47.293703+0000","last_undegraded":"2024-04-26T19:59:47.293703+0000","last_fullsized":"2024-04-26T19:59:47.293703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464282+0000","last_change":"2024-04-26T19:59:46.290194+0000","last_active":"2024-04-26T19:59:47.464282+0000","last_peered":"2024-04-26T19:59:47.464282+0000","last_clean":"2024-04-26T19:59:47.464282+0000","last_became_active":"2024-04-26T19:59:46.289473+0000","last_became_peered":"2024-04-26T19:59:46.289473+0000","last_unstale":"2024-04-26T19:59:47.464282+0000","last_undegraded":"2024-04-26T19:59:47.464282+0000","last_fullsized":"2024-04-26T19:59:47.464282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468794+0000","last_change":"2024-04-26T19:59:46.312802+0000","last_active":"2024-04-26T19:59:47.468794+0000","last_peered":"2024-04-26T19:59:47.468794+0000","last_clean":"2024-04-26T19:59:47.468794+0000","last_became_active":"2024-04-26T19:59:46.312497+0000","last_became_peered":"2024-04-26T19:59:46.312497+0000","last_unstale":"2024-04-26T19:59:47.468794+0000","last_undegraded":"2024-04-26T19:59:47.468794+0000","last_fullsized":"2024-04-26T19:59:47.468794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297946+0000","last_change":"2024-04-26T19:59:46.290353+0000","last_active":"2024-04-26T19:59:47.297946+0000","last_peered":"2024-04-26T19:59:47.297946+0000","last_clean":"2024-04-26T19:59:47.297946+0000","last_became_active":"2024-04-26T19:59:46.289992+0000","last_became_peered":"2024-04-26T19:59:46.289992+0000","last_unstale":"2024-04-26T19:59:47.297946+0000","last_undegraded":"2024-04-26T19:59:47.297946+0000","last_fullsized":"2024-04-26T19:59:47.297946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468211+0000","last_change":"2024-04-26T19:59:46.302429+0000","last_active":"2024-04-26T19:59:47.468211+0000","last_peered":"2024-04-26T19:59:47.468211+0000","last_clean":"2024-04-26T19:59:47.468211+0000","last_became_active":"2024-04-26T19:59:46.302123+0000","last_became_peered":"2024-04-26T19:59:46.302123+0000","last_unstale":"2024-04-26T19:59:47.468211+0000","last_undegraded":"2024-04-26T19:59:47.468211+0000","last_fullsized":"2024-04-26T19:59:47.468211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468516+0000","last_change":"2024-04-26T19:59:46.300571+0000","last_active":"2024-04-26T19:59:47.468516+0000","last_peered":"2024-04-26T19:59:47.468516+0000","last_clean":"2024-04-26T19:59:47.468516+0000","last_became_active":"2024-04-26T19:59:46.300247+0000","last_became_peered":"2024-04-26T19:59:46.300247+0000","last_unstale":"2024-04-26T19:59:47.468516+0000","last_undegraded":"2024-04-26T19:59:47.468516+0000","last_fullsized":"2024-04-26T19:59:47.468516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464971+0000","last_change":"2024-04-26T19:59:46.300279+0000","last_active":"2024-04-26T19:59:47.464971+0000","last_peered":"2024-04-26T19:59:47.464971+0000","last_clean":"2024-04-26T19:59:47.464971+0000","last_became_active":"2024-04-26T19:59:46.300053+0000","last_became_peered":"2024-04-26T19:59:46.300053+0000","last_unstale":"2024-04-26T19:59:47.464971+0000","last_undegraded":"2024-04-26T19:59:47.464971+0000","last_fullsized":"2024-04-26T19:59:47.464971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466037+0000","last_change":"2024-04-26T19:59:46.305198+0000","last_active":"2024-04-26T19:59:47.466037+0000","last_peered":"2024-04-26T19:59:47.466037+0000","last_clean":"2024-04-26T19:59:47.466037+0000","last_became_active":"2024-04-26T19:59:46.303321+0000","last_became_peered":"2024-04-26T19:59:46.303321+0000","last_unstale":"2024-04-26T19:59:47.466037+0000","last_undegraded":"2024-04-26T19:59:47.466037+0000","last_fullsized":"2024-04-26T19:59:47.466037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463271+0000","last_change":"2024-04-26T19:59:46.313226+0000","last_active":"2024-04-26T19:59:47.463271+0000","last_peered":"2024-04-26T19:59:47.463271+0000","last_clean":"2024-04-26T19:59:47.463271+0000","last_became_active":"2024-04-26T19:59:46.312933+0000","last_became_peered":"2024-04-26T19:59:46.312933+0000","last_unstale":"2024-04-26T19:59:47.463271+0000","last_undegraded":"2024-04-26T19:59:47.463271+0000","last_fullsized":"2024-04-26T19:59:47.463271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464413+0000","last_change":"2024-04-26T19:59:46.296413+0000","last_active":"2024-04-26T19:59:47.464413+0000","last_peered":"2024-04-26T19:59:47.464413+0000","last_clean":"2024-04-26T19:59:47.464413+0000","last_became_active":"2024-04-26T19:59:46.295760+0000","last_became_peered":"2024-04-26T19:59:46.295760+0000","last_unstale":"2024-04-26T19:59:47.464413+0000","last_undegraded":"2024-04-26T19:59:47.464413+0000","last_fullsized":"2024-04-26T19:59:47.464413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465037+0000","last_change":"2024-04-26T19:59:46.314022+0000","last_active":"2024-04-26T19:59:47.465037+0000","last_peered":"2024-04-26T19:59:47.465037+0000","last_clean":"2024-04-26T19:59:47.465037+0000","last_became_active":"2024-04-26T19:59:46.313768+0000","last_became_peered":"2024-04-26T19:59:46.313768+0000","last_unstale":"2024-04-26T19:59:47.465037+0000","last_undegraded":"2024-04-26T19:59:47.465037+0000","last_fullsized":"2024-04-26T19:59:47.465037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469629+0000","last_change":"2024-04-26T19:59:46.299335+0000","last_active":"2024-04-26T19:59:47.469629+0000","last_peered":"2024-04-26T19:59:47.469629+0000","last_clean":"2024-04-26T19:59:47.469629+0000","last_became_active":"2024-04-26T19:59:46.299048+0000","last_became_peered":"2024-04-26T19:59:46.299048+0000","last_unstale":"2024-04-26T19:59:47.469629+0000","last_undegraded":"2024-04-26T19:59:47.469629+0000","last_fullsized":"2024-04-26T19:59:47.469629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294167+0000","last_change":"2024-04-26T19:59:46.297650+0000","last_active":"2024-04-26T19:59:47.294167+0000","last_peered":"2024-04-26T19:59:47.294167+0000","last_clean":"2024-04-26T19:59:47.294167+0000","last_became_active":"2024-04-26T19:59:46.297368+0000","last_became_peered":"2024-04-26T19:59:46.297368+0000","last_unstale":"2024-04-26T19:59:47.294167+0000","last_undegraded":"2024-04-26T19:59:47.294167+0000","last_fullsized":"2024-04-26T19:59:47.294167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464415+0000","last_change":"2024-04-26T19:59:46.313265+0000","last_active":"2024-04-26T19:59:47.464415+0000","last_peered":"2024-04-26T19:59:47.464415+0000","last_clean":"2024-04-26T19:59:47.464415+0000","last_became_active":"2024-04-26T19:59:46.313035+0000","last_became_peered":"2024-04-26T19:59:46.313035+0000","last_unstale":"2024-04-26T19:59:47.464415+0000","last_undegraded":"2024-04-26T19:59:47.464415+0000","last_fullsized":"2024-04-26T19:59:47.464415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465117+0000","last_change":"2024-04-26T19:59:46.294814+0000","last_active":"2024-04-26T19:59:47.465117+0000","last_peered":"2024-04-26T19:59:47.465117+0000","last_clean":"2024-04-26T19:59:47.465117+0000","last_became_active":"2024-04-26T19:59:46.294277+0000","last_became_peered":"2024-04-26T19:59:46.294277+0000","last_unstale":"2024-04-26T19:59:47.465117+0000","last_undegraded":"2024-04-26T19:59:47.465117+0000","last_fullsized":"2024-04-26T19:59:47.465117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466341+0000","last_change":"2024-04-26T19:59:46.297299+0000","last_active":"2024-04-26T19:59:47.466341+0000","last_peered":"2024-04-26T19:59:47.466341+0000","last_clean":"2024-04-26T19:59:47.466341+0000","last_became_active":"2024-04-26T19:59:46.296375+0000","last_became_peered":"2024-04-26T19:59:46.296375+0000","last_unstale":"2024-04-26T19:59:47.466341+0000","last_undegraded":"2024-04-26T19:59:47.466341+0000","last_fullsized":"2024-04-26T19:59:47.466341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297656+0000","last_change":"2024-04-26T19:59:46.292257+0000","last_active":"2024-04-26T19:59:47.297656+0000","last_peered":"2024-04-26T19:59:47.297656+0000","last_clean":"2024-04-26T19:59:47.297656+0000","last_became_active":"2024-04-26T19:59:46.291980+0000","last_became_peered":"2024-04-26T19:59:46.291980+0000","last_unstale":"2024-04-26T19:59:47.297656+0000","last_undegraded":"2024-04-26T19:59:47.297656+0000","last_fullsized":"2024-04-26T19:59:47.297656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464925+0000","last_change":"2024-04-26T19:59:46.283803+0000","last_active":"2024-04-26T19:59:47.464925+0000","last_peered":"2024-04-26T19:59:47.464925+0000","last_clean":"2024-04-26T19:59:47.464925+0000","last_became_active":"2024-04-26T19:59:46.283455+0000","last_became_peered":"2024-04-26T19:59:46.283455+0000","last_unstale":"2024-04-26T19:59:47.464925+0000","last_undegraded":"2024-04-26T19:59:47.464925+0000","last_fullsized":"2024-04-26T19:59:47.464925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297641+0000","last_change":"2024-04-26T19:59:46.290324+0000","last_active":"2024-04-26T19:59:47.297641+0000","last_peered":"2024-04-26T19:59:47.297641+0000","last_clean":"2024-04-26T19:59:47.297641+0000","last_became_active":"2024-04-26T19:59:46.289926+0000","last_became_peered":"2024-04-26T19:59:46.289926+0000","last_unstale":"2024-04-26T19:59:47.297641+0000","last_undegraded":"2024-04-26T19:59:47.297641+0000","last_fullsized":"2024-04-26T19:59:47.297641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:56.862425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464081+0000","last_change":"2024-04-26T19:59:45.291916+0000","last_active":"2024-04-26T19:59:47.464081+0000","last_peered":"2024-04-26T19:59:47.464081+0000","last_clean":"2024-04-26T19:59:47.464081+0000","last_became_active":"2024-04-26T19:59:45.291242+0000","last_became_peered":"2024-04-26T19:59:45.291242+0000","last_unstale":"2024-04-26T19:59:47.464081+0000","last_undegraded":"2024-04-26T19:59:47.464081+0000","last_fullsized":"2024-04-26T19:59:47.464081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465265+0000","last_change":"2024-04-26T19:59:46.295081+0000","last_active":"2024-04-26T19:59:47.465265+0000","last_peered":"2024-04-26T19:59:47.465265+0000","last_clean":"2024-04-26T19:59:47.465265+0000","last_became_active":"2024-04-26T19:59:46.294800+0000","last_became_peered":"2024-04-26T19:59:46.294800+0000","last_unstale":"2024-04-26T19:59:47.465265+0000","last_undegraded":"2024-04-26T19:59:47.465265+0000","last_fullsized":"2024-04-26T19:59:47.465265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465946+0000","last_change":"2024-04-26T19:59:45.286181+0000","last_active":"2024-04-26T19:59:47.465946+0000","last_peered":"2024-04-26T19:59:47.465946+0000","last_clean":"2024-04-26T19:59:47.465946+0000","last_became_active":"2024-04-26T19:59:45.285851+0000","last_became_peered":"2024-04-26T19:59:45.285851+0000","last_unstale":"2024-04-26T19:59:47.465946+0000","last_undegraded":"2024-04-26T19:59:47.465946+0000","last_fullsized":"2024-04-26T19:59:47.465946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463328+0000","last_change":"2024-04-26T19:59:45.286697+0000","last_active":"2024-04-26T19:59:47.463328+0000","last_peered":"2024-04-26T19:59:47.463328+0000","last_clean":"2024-04-26T19:59:47.463328+0000","last_became_active":"2024-04-26T19:59:45.286293+0000","last_became_peered":"2024-04-26T19:59:45.286293+0000","last_unstale":"2024-04-26T19:59:47.463328+0000","last_undegraded":"2024-04-26T19:59:47.463328+0000","last_fullsized":"2024-04-26T19:59:47.463328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464589+0000","last_change":"2024-04-26T19:59:46.294713+0000","last_active":"2024-04-26T19:59:47.464589+0000","last_peered":"2024-04-26T19:59:47.464589+0000","last_clean":"2024-04-26T19:59:47.464589+0000","last_became_active":"2024-04-26T19:59:46.294022+0000","last_became_peered":"2024-04-26T19:59:46.294022+0000","last_unstale":"2024-04-26T19:59:47.464589+0000","last_undegraded":"2024-04-26T19:59:47.464589+0000","last_fullsized":"2024-04-26T19:59:47.464589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464432+0000","last_change":"2024-04-26T19:59:45.284101+0000","last_active":"2024-04-26T19:59:47.464432+0000","last_peered":"2024-04-26T19:59:47.464432+0000","last_clean":"2024-04-26T19:59:47.464432+0000","last_became_active":"2024-04-26T19:59:45.283784+0000","last_became_peered":"2024-04-26T19:59:45.283784+0000","last_unstale":"2024-04-26T19:59:47.464432+0000","last_undegraded":"2024-04-26T19:59:47.464432+0000","last_fullsized":"2024-04-26T19:59:47.464432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293526+0000","last_change":"2024-04-26T19:59:46.296659+0000","last_active":"2024-04-26T19:59:47.293526+0000","last_peered":"2024-04-26T19:59:47.293526+0000","last_clean":"2024-04-26T19:59:47.293526+0000","last_became_active":"2024-04-26T19:59:46.296362+0000","last_became_peered":"2024-04-26T19:59:46.296362+0000","last_unstale":"2024-04-26T19:59:47.293526+0000","last_undegraded":"2024-04-26T19:59:47.293526+0000","last_fullsized":"2024-04-26T19:59:47.293526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468741+0000","last_change":"2024-04-26T19:59:45.299230+0000","last_active":"2024-04-26T19:59:47.468741+0000","last_peered":"2024-04-26T19:59:47.468741+0000","last_clean":"2024-04-26T19:59:47.468741+0000","last_became_active":"2024-04-26T19:59:45.298903+0000","last_became_peered":"2024-04-26T19:59:45.298903+0000","last_unstale":"2024-04-26T19:59:47.468741+0000","last_undegraded":"2024-04-26T19:59:47.468741+0000","last_fullsized":"2024-04-26T19:59:47.468741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467717+0000","last_change":"2024-04-26T19:59:46.306993+0000","last_active":"2024-04-26T19:59:47.467717+0000","last_peered":"2024-04-26T19:59:47.467717+0000","last_clean":"2024-04-26T19:59:47.467717+0000","last_became_active":"2024-04-26T19:59:46.305722+0000","last_became_peered":"2024-04-26T19:59:46.305722+0000","last_unstale":"2024-04-26T19:59:47.467717+0000","last_undegraded":"2024-04-26T19:59:47.467717+0000","last_fullsized":"2024-04-26T19:59:47.467717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467742+0000","last_change":"2024-04-26T19:59:45.297752+0000","last_active":"2024-04-26T19:59:47.467742+0000","last_peered":"2024-04-26T19:59:47.467742+0000","last_clean":"2024-04-26T19:59:47.467742+0000","last_became_active":"2024-04-26T19:59:45.297462+0000","last_became_peered":"2024-04-26T19:59:45.297462+0000","last_unstale":"2024-04-26T19:59:47.467742+0000","last_undegraded":"2024-04-26T19:59:47.467742+0000","last_fullsized":"2024-04-26T19:59:47.467742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464194+0000","last_change":"2024-04-26T19:59:46.290536+0000","last_active":"2024-04-26T19:59:47.464194+0000","last_peered":"2024-04-26T19:59:47.464194+0000","last_clean":"2024-04-26T19:59:47.464194+0000","last_became_active":"2024-04-26T19:59:46.289737+0000","last_became_peered":"2024-04-26T19:59:46.289737+0000","last_unstale":"2024-04-26T19:59:47.464194+0000","last_undegraded":"2024-04-26T19:59:47.464194+0000","last_fullsized":"2024-04-26T19:59:47.464194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294232+0000","last_change":"2024-04-26T19:59:45.293462+0000","last_active":"2024-04-26T19:59:47.294232+0000","last_peered":"2024-04-26T19:59:47.294232+0000","last_clean":"2024-04-26T19:59:47.294232+0000","last_became_active":"2024-04-26T19:59:45.292997+0000","last_became_peered":"2024-04-26T19:59:45.292997+0000","last_unstale":"2024-04-26T19:59:47.294232+0000","last_undegraded":"2024-04-26T19:59:47.294232+0000","last_fullsized":"2024-04-26T19:59:47.294232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298199+0000","last_change":"2024-04-26T19:59:46.307369+0000","last_active":"2024-04-26T19:59:47.298199+0000","last_peered":"2024-04-26T19:59:47.298199+0000","last_clean":"2024-04-26T19:59:47.298199+0000","last_became_active":"2024-04-26T19:59:46.306973+0000","last_became_peered":"2024-04-26T19:59:46.306973+0000","last_unstale":"2024-04-26T19:59:47.298199+0000","last_undegraded":"2024-04-26T19:59:47.298199+0000","last_fullsized":"2024-04-26T19:59:47.298199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469706+0000","last_change":"2024-04-26T19:59:45.292264+0000","last_active":"2024-04-26T19:59:47.469706+0000","last_peered":"2024-04-26T19:59:47.469706+0000","last_clean":"2024-04-26T19:59:47.469706+0000","last_became_active":"2024-04-26T19:59:45.290211+0000","last_became_peered":"2024-04-26T19:59:45.290211+0000","last_unstale":"2024-04-26T19:59:47.469706+0000","last_undegraded":"2024-04-26T19:59:47.469706+0000","last_fullsized":"2024-04-26T19:59:47.469706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467202+0000","last_change":"2024-04-26T19:59:46.294340+0000","last_active":"2024-04-26T19:59:47.467202+0000","last_peered":"2024-04-26T19:59:47.467202+0000","last_clean":"2024-04-26T19:59:47.467202+0000","last_became_active":"2024-04-26T19:59:46.294017+0000","last_became_peered":"2024-04-26T19:59:46.294017+0000","last_unstale":"2024-04-26T19:59:47.467202+0000","last_undegraded":"2024-04-26T19:59:47.467202+0000","last_fullsized":"2024-04-26T19:59:47.467202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465254+0000","last_change":"2024-04-26T19:59:45.285166+0000","last_active":"2024-04-26T19:59:47.465254+0000","last_peered":"2024-04-26T19:59:47.465254+0000","last_clean":"2024-04-26T19:59:47.465254+0000","last_became_active":"2024-04-26T19:59:45.284863+0000","last_became_peered":"2024-04-26T19:59:45.284863+0000","last_unstale":"2024-04-26T19:59:47.465254+0000","last_undegraded":"2024-04-26T19:59:47.465254+0000","last_fullsized":"2024-04-26T19:59:47.465254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464651+0000","last_change":"2024-04-26T19:59:46.296940+0000","last_active":"2024-04-26T19:59:47.464651+0000","last_peered":"2024-04-26T19:59:47.464651+0000","last_clean":"2024-04-26T19:59:47.464651+0000","last_became_active":"2024-04-26T19:59:46.296695+0000","last_became_peered":"2024-04-26T19:59:46.296695+0000","last_unstale":"2024-04-26T19:59:47.464651+0000","last_undegraded":"2024-04-26T19:59:47.464651+0000","last_fullsized":"2024-04-26T19:59:47.464651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466255+0000","last_change":"2024-04-26T19:59:45.288319+0000","last_active":"2024-04-26T19:59:47.466255+0000","last_peered":"2024-04-26T19:59:47.466255+0000","last_clean":"2024-04-26T19:59:47.466255+0000","last_became_active":"2024-04-26T19:59:45.287908+0000","last_became_peered":"2024-04-26T19:59:45.287908+0000","last_unstale":"2024-04-26T19:59:47.466255+0000","last_undegraded":"2024-04-26T19:59:47.466255+0000","last_fullsized":"2024-04-26T19:59:47.466255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469394+0000","last_change":"2024-04-26T19:59:46.314258+0000","last_active":"2024-04-26T19:59:47.469394+0000","last_peered":"2024-04-26T19:59:47.469394+0000","last_clean":"2024-04-26T19:59:47.469394+0000","last_became_active":"2024-04-26T19:59:46.313941+0000","last_became_peered":"2024-04-26T19:59:46.313941+0000","last_unstale":"2024-04-26T19:59:47.469394+0000","last_undegraded":"2024-04-26T19:59:47.469394+0000","last_fullsized":"2024-04-26T19:59:47.469394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466928+0000","last_change":"2024-04-26T19:59:45.292499+0000","last_active":"2024-04-26T19:59:47.466928+0000","last_peered":"2024-04-26T19:59:47.466928+0000","last_clean":"2024-04-26T19:59:47.466928+0000","last_became_active":"2024-04-26T19:59:45.288991+0000","last_became_peered":"2024-04-26T19:59:45.288991+0000","last_unstale":"2024-04-26T19:59:47.466928+0000","last_undegraded":"2024-04-26T19:59:47.466928+0000","last_fullsized":"2024-04-26T19:59:47.466928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465852+0000","last_change":"2024-04-26T19:59:46.283597+0000","last_active":"2024-04-26T19:59:47.465852+0000","last_peered":"2024-04-26T19:59:47.465852+0000","last_clean":"2024-04-26T19:59:47.465852+0000","last_became_active":"2024-04-26T19:59:46.283235+0000","last_became_peered":"2024-04-26T19:59:46.283235+0000","last_unstale":"2024-04-26T19:59:47.465852+0000","last_undegraded":"2024-04-26T19:59:47.465852+0000","last_fullsized":"2024-04-26T19:59:47.465852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297270+0000","last_change":"2024-04-26T19:59:45.276538+0000","last_active":"2024-04-26T19:59:47.297270+0000","last_peered":"2024-04-26T19:59:47.297270+0000","last_clean":"2024-04-26T19:59:47.297270+0000","last_became_active":"2024-04-26T19:59:45.276197+0000","last_became_peered":"2024-04-26T19:59:45.276197+0000","last_unstale":"2024-04-26T19:59:47.297270+0000","last_undegraded":"2024-04-26T19:59:47.297270+0000","last_fullsized":"2024-04-26T19:59:47.297270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466760+0000","last_change":"2024-04-26T19:59:46.314180+0000","last_active":"2024-04-26T19:59:47.466760+0000","last_peered":"2024-04-26T19:59:47.466760+0000","last_clean":"2024-04-26T19:59:47.466760+0000","last_became_active":"2024-04-26T19:59:46.313845+0000","last_became_peered":"2024-04-26T19:59:46.313845+0000","last_unstale":"2024-04-26T19:59:47.466760+0000","last_undegraded":"2024-04-26T19:59:47.466760+0000","last_fullsized":"2024-04-26T19:59:47.466760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298083+0000","last_change":"2024-04-26T19:59:45.282738+0000","last_active":"2024-04-26T19:59:47.298083+0000","last_peered":"2024-04-26T19:59:47.298083+0000","last_clean":"2024-04-26T19:59:47.298083+0000","last_became_active":"2024-04-26T19:59:45.281228+0000","last_became_peered":"2024-04-26T19:59:45.281228+0000","last_unstale":"2024-04-26T19:59:47.298083+0000","last_undegraded":"2024-04-26T19:59:47.298083+0000","last_fullsized":"2024-04-26T19:59:47.298083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465206+0000","last_change":"2024-04-26T19:59:46.314419+0000","last_active":"2024-04-26T19:59:47.465206+0000","last_peered":"2024-04-26T19:59:47.465206+0000","last_clean":"2024-04-26T19:59:47.465206+0000","last_became_active":"2024-04-26T19:59:46.313946+0000","last_became_peered":"2024-04-26T19:59:46.313946+0000","last_unstale":"2024-04-26T19:59:47.465206+0000","last_undegraded":"2024-04-26T19:59:47.465206+0000","last_fullsized":"2024-04-26T19:59:47.465206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297898+0000","last_change":"2024-04-26T19:59:45.266464+0000","last_active":"2024-04-26T19:59:47.297898+0000","last_peered":"2024-04-26T19:59:47.297898+0000","last_clean":"2024-04-26T19:59:47.297898+0000","last_became_active":"2024-04-26T19:59:45.265937+0000","last_became_peered":"2024-04-26T19:59:45.265937+0000","last_unstale":"2024-04-26T19:59:47.297898+0000","last_undegraded":"2024-04-26T19:59:47.297898+0000","last_fullsized":"2024-04-26T19:59:47.297898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:00:35.753658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465838+0000","last_change":"2024-04-26T19:59:46.314568+0000","last_active":"2024-04-26T19:59:47.465838+0000","last_peered":"2024-04-26T19:59:47.465838+0000","last_clean":"2024-04-26T19:59:47.465838+0000","last_became_active":"2024-04-26T19:59:46.313865+0000","last_became_peered":"2024-04-26T19:59:46.313865+0000","last_unstale":"2024-04-26T19:59:47.465838+0000","last_undegraded":"2024-04-26T19:59:47.465838+0000","last_fullsized":"2024-04-26T19:59:47.465838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294351+0000","last_change":"2024-04-26T19:59:45.281434+0000","last_active":"2024-04-26T19:59:47.294351+0000","last_peered":"2024-04-26T19:59:47.294351+0000","last_clean":"2024-04-26T19:59:47.294351+0000","last_became_active":"2024-04-26T19:59:45.280816+0000","last_became_peered":"2024-04-26T19:59:45.280816+0000","last_unstale":"2024-04-26T19:59:47.294351+0000","last_undegraded":"2024-04-26T19:59:47.294351+0000","last_fullsized":"2024-04-26T19:59:47.294351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466518+0000","last_change":"2024-04-26T19:59:46.295918+0000","last_active":"2024-04-26T19:59:47.466518+0000","last_peered":"2024-04-26T19:59:47.466518+0000","last_clean":"2024-04-26T19:59:47.466518+0000","last_became_active":"2024-04-26T19:59:46.295630+0000","last_became_peered":"2024-04-26T19:59:46.295630+0000","last_unstale":"2024-04-26T19:59:47.466518+0000","last_undegraded":"2024-04-26T19:59:47.466518+0000","last_fullsized":"2024-04-26T19:59:47.466518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297781+0000","last_change":"2024-04-26T19:59:45.288285+0000","last_active":"2024-04-26T19:59:47.297781+0000","last_peered":"2024-04-26T19:59:47.297781+0000","last_clean":"2024-04-26T19:59:47.297781+0000","last_became_active":"2024-04-26T19:59:45.287962+0000","last_became_peered":"2024-04-26T19:59:45.287962+0000","last_unstale":"2024-04-26T19:59:47.297781+0000","last_undegraded":"2024-04-26T19:59:47.297781+0000","last_fullsized":"2024-04-26T19:59:47.297781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467597+0000","last_change":"2024-04-26T19:59:46.310806+0000","last_active":"2024-04-26T19:59:47.467597+0000","last_peered":"2024-04-26T19:59:47.467597+0000","last_clean":"2024-04-26T19:59:47.467597+0000","last_became_active":"2024-04-26T19:59:46.310281+0000","last_became_peered":"2024-04-26T19:59:46.310281+0000","last_unstale":"2024-04-26T19:59:47.467597+0000","last_undegraded":"2024-04-26T19:59:47.467597+0000","last_fullsized":"2024-04-26T19:59:47.467597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464724+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T19:59:47.464724+0000","last_peered":"2024-04-26T19:59:47.464724+0000","last_clean":"2024-04-26T19:59:47.464724+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T19:59:47.464724+0000","last_undegraded":"2024-04-26T19:59:47.464724+0000","last_fullsized":"2024-04-26T19:59:47.464724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465285+0000","last_change":"2024-04-26T19:59:45.281392+0000","last_active":"2024-04-26T19:59:47.465285+0000","last_peered":"2024-04-26T19:59:47.465285+0000","last_clean":"2024-04-26T19:59:47.465285+0000","last_became_active":"2024-04-26T19:59:45.281041+0000","last_became_peered":"2024-04-26T19:59:45.281041+0000","last_unstale":"2024-04-26T19:59:47.465285+0000","last_undegraded":"2024-04-26T19:59:47.465285+0000","last_fullsized":"2024-04-26T19:59:47.465285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465458+0000","last_change":"2024-04-26T19:59:46.294932+0000","last_active":"2024-04-26T19:59:47.465458+0000","last_peered":"2024-04-26T19:59:47.465458+0000","last_clean":"2024-04-26T19:59:47.465458+0000","last_became_active":"2024-04-26T19:59:46.294670+0000","last_became_peered":"2024-04-26T19:59:46.294670+0000","last_unstale":"2024-04-26T19:59:47.465458+0000","last_undegraded":"2024-04-26T19:59:47.465458+0000","last_fullsized":"2024-04-26T19:59:47.465458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464419+0000","last_change":"2024-04-26T19:59:45.299670+0000","last_active":"2024-04-26T19:59:47.464419+0000","last_peered":"2024-04-26T19:59:47.464419+0000","last_clean":"2024-04-26T19:59:47.464419+0000","last_became_active":"2024-04-26T19:59:45.299344+0000","last_became_peered":"2024-04-26T19:59:45.299344+0000","last_unstale":"2024-04-26T19:59:47.464419+0000","last_undegraded":"2024-04-26T19:59:47.464419+0000","last_fullsized":"2024-04-26T19:59:47.464419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465122+0000","last_change":"2024-04-26T19:59:46.300383+0000","last_active":"2024-04-26T19:59:47.465122+0000","last_peered":"2024-04-26T19:59:47.465122+0000","last_clean":"2024-04-26T19:59:47.465122+0000","last_became_active":"2024-04-26T19:59:46.299990+0000","last_became_peered":"2024-04-26T19:59:46.299990+0000","last_unstale":"2024-04-26T19:59:47.465122+0000","last_undegraded":"2024-04-26T19:59:47.465122+0000","last_fullsized":"2024-04-26T19:59:47.465122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297425+0000","last_change":"2024-04-26T19:59:45.270294+0000","last_active":"2024-04-26T19:59:47.297425+0000","last_peered":"2024-04-26T19:59:47.297425+0000","last_clean":"2024-04-26T19:59:47.297425+0000","last_became_active":"2024-04-26T19:59:45.269847+0000","last_became_peered":"2024-04-26T19:59:45.269847+0000","last_unstale":"2024-04-26T19:59:47.297425+0000","last_undegraded":"2024-04-26T19:59:47.297425+0000","last_fullsized":"2024-04-26T19:59:47.297425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294086+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.294086+0000","last_peered":"2024-04-26T19:59:47.294086+0000","last_clean":"2024-04-26T19:59:47.294086+0000","last_became_active":"2024-04-26T19:59:46.290208+0000","last_became_peered":"2024-04-26T19:59:46.290208+0000","last_unstale":"2024-04-26T19:59:47.294086+0000","last_undegraded":"2024-04-26T19:59:47.294086+0000","last_fullsized":"2024-04-26T19:59:47.294086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466469+0000","last_change":"2024-04-26T19:59:45.292265+0000","last_active":"2024-04-26T19:59:47.466469+0000","last_peered":"2024-04-26T19:59:47.466469+0000","last_clean":"2024-04-26T19:59:47.466469+0000","last_became_active":"2024-04-26T19:59:45.291785+0000","last_became_peered":"2024-04-26T19:59:45.291785+0000","last_unstale":"2024-04-26T19:59:47.466469+0000","last_undegraded":"2024-04-26T19:59:47.466469+0000","last_fullsized":"2024-04-26T19:59:47.466469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297995+0000","last_change":"2024-04-26T19:59:46.294373+0000","last_active":"2024-04-26T19:59:47.297995+0000","last_peered":"2024-04-26T19:59:47.297995+0000","last_clean":"2024-04-26T19:59:47.297995+0000","last_became_active":"2024-04-26T19:59:46.292939+0000","last_became_peered":"2024-04-26T19:59:46.292939+0000","last_unstale":"2024-04-26T19:59:47.297995+0000","last_undegraded":"2024-04-26T19:59:47.297995+0000","last_fullsized":"2024-04-26T19:59:47.297995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294407+0000","last_change":"2024-04-26T19:59:45.286062+0000","last_active":"2024-04-26T19:59:47.294407+0000","last_peered":"2024-04-26T19:59:47.294407+0000","last_clean":"2024-04-26T19:59:47.294407+0000","last_became_active":"2024-04-26T19:59:45.282680+0000","last_became_peered":"2024-04-26T19:59:45.282680+0000","last_unstale":"2024-04-26T19:59:47.294407+0000","last_undegraded":"2024-04-26T19:59:47.294407+0000","last_fullsized":"2024-04-26T19:59:47.294407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467709+0000","last_change":"2024-04-26T19:59:46.303855+0000","last_active":"2024-04-26T19:59:47.467709+0000","last_peered":"2024-04-26T19:59:47.467709+0000","last_clean":"2024-04-26T19:59:47.467709+0000","last_became_active":"2024-04-26T19:59:46.298323+0000","last_became_peered":"2024-04-26T19:59:46.298323+0000","last_unstale":"2024-04-26T19:59:47.467709+0000","last_undegraded":"2024-04-26T19:59:47.467709+0000","last_fullsized":"2024-04-26T19:59:47.467709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468317+0000","last_change":"2024-04-26T19:59:45.302616+0000","last_active":"2024-04-26T19:59:47.468317+0000","last_peered":"2024-04-26T19:59:47.468317+0000","last_clean":"2024-04-26T19:59:47.468317+0000","last_became_active":"2024-04-26T19:59:45.302164+0000","last_became_peered":"2024-04-26T19:59:45.302164+0000","last_unstale":"2024-04-26T19:59:47.468317+0000","last_undegraded":"2024-04-26T19:59:47.468317+0000","last_fullsized":"2024-04-26T19:59:47.468317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294273+0000","last_change":"2024-04-26T19:59:46.290753+0000","last_active":"2024-04-26T19:59:47.294273+0000","last_peered":"2024-04-26T19:59:47.294273+0000","last_clean":"2024-04-26T19:59:47.294273+0000","last_became_active":"2024-04-26T19:59:46.290477+0000","last_became_peered":"2024-04-26T19:59:46.290477+0000","last_unstale":"2024-04-26T19:59:47.294273+0000","last_undegraded":"2024-04-26T19:59:47.294273+0000","last_fullsized":"2024-04-26T19:59:47.294273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294611+0000","last_change":"2024-04-26T19:59:45.297751+0000","last_active":"2024-04-26T19:59:47.294611+0000","last_peered":"2024-04-26T19:59:47.294611+0000","last_clean":"2024-04-26T19:59:47.294611+0000","last_became_active":"2024-04-26T19:59:45.297426+0000","last_became_peered":"2024-04-26T19:59:45.297426+0000","last_unstale":"2024-04-26T19:59:47.294611+0000","last_undegraded":"2024-04-26T19:59:47.294611+0000","last_fullsized":"2024-04-26T19:59:47.294611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465393+0000","last_change":"2024-04-26T19:59:46.298839+0000","last_active":"2024-04-26T19:59:47.465393+0000","last_peered":"2024-04-26T19:59:47.465393+0000","last_clean":"2024-04-26T19:59:47.465393+0000","last_became_active":"2024-04-26T19:59:46.298552+0000","last_became_peered":"2024-04-26T19:59:46.298552+0000","last_unstale":"2024-04-26T19:59:47.465393+0000","last_undegraded":"2024-04-26T19:59:47.465393+0000","last_fullsized":"2024-04-26T19:59:47.465393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468933+0000","last_change":"2024-04-26T19:59:45.291668+0000","last_active":"2024-04-26T19:59:47.468933+0000","last_peered":"2024-04-26T19:59:47.468933+0000","last_clean":"2024-04-26T19:59:47.468933+0000","last_became_active":"2024-04-26T19:59:45.291333+0000","last_became_peered":"2024-04-26T19:59:45.291333+0000","last_unstale":"2024-04-26T19:59:47.468933+0000","last_undegraded":"2024-04-26T19:59:47.468933+0000","last_fullsized":"2024-04-26T19:59:47.468933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465754+0000","last_change":"2024-04-26T19:59:46.276488+0000","last_active":"2024-04-26T19:59:47.465754+0000","last_peered":"2024-04-26T19:59:47.465754+0000","last_clean":"2024-04-26T19:59:47.465754+0000","last_became_active":"2024-04-26T19:59:46.274349+0000","last_became_peered":"2024-04-26T19:59:46.274349+0000","last_unstale":"2024-04-26T19:59:47.465754+0000","last_undegraded":"2024-04-26T19:59:47.465754+0000","last_fullsized":"2024-04-26T19:59:47.465754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464100+0000","last_change":"2024-04-26T19:59:45.300210+0000","last_active":"2024-04-26T19:59:47.464100+0000","last_peered":"2024-04-26T19:59:47.464100+0000","last_clean":"2024-04-26T19:59:47.464100+0000","last_became_active":"2024-04-26T19:59:45.299954+0000","last_became_peered":"2024-04-26T19:59:45.299954+0000","last_unstale":"2024-04-26T19:59:47.464100+0000","last_undegraded":"2024-04-26T19:59:47.464100+0000","last_fullsized":"2024-04-26T19:59:47.464100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297802+0000","last_change":"2024-04-26T19:59:46.299193+0000","last_active":"2024-04-26T19:59:47.297802+0000","last_peered":"2024-04-26T19:59:47.297802+0000","last_clean":"2024-04-26T19:59:47.297802+0000","last_became_active":"2024-04-26T19:59:46.298053+0000","last_became_peered":"2024-04-26T19:59:46.298053+0000","last_unstale":"2024-04-26T19:59:47.297802+0000","last_undegraded":"2024-04-26T19:59:47.297802+0000","last_fullsized":"2024-04-26T19:59:47.297802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:18.579103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293761+0000","last_change":"2024-04-26T19:59:45.290679+0000","last_active":"2024-04-26T19:59:47.293761+0000","last_peered":"2024-04-26T19:59:47.293761+0000","last_clean":"2024-04-26T19:59:47.293761+0000","last_became_active":"2024-04-26T19:59:45.290411+0000","last_became_peered":"2024-04-26T19:59:45.290411+0000","last_unstale":"2024-04-26T19:59:47.293761+0000","last_undegraded":"2024-04-26T19:59:47.293761+0000","last_fullsized":"2024-04-26T19:59:47.293761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464967+0000","last_change":"2024-04-26T19:59:46.288442+0000","last_active":"2024-04-26T19:59:47.464967+0000","last_peered":"2024-04-26T19:59:47.464967+0000","last_clean":"2024-04-26T19:59:47.464967+0000","last_became_active":"2024-04-26T19:59:46.288239+0000","last_became_peered":"2024-04-26T19:59:46.288239+0000","last_unstale":"2024-04-26T19:59:47.464967+0000","last_undegraded":"2024-04-26T19:59:47.464967+0000","last_fullsized":"2024-04-26T19:59:47.464967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294034+0000","last_change":"2024-04-26T19:59:45.280986+0000","last_active":"2024-04-26T19:59:47.294034+0000","last_peered":"2024-04-26T19:59:47.294034+0000","last_clean":"2024-04-26T19:59:47.294034+0000","last_became_active":"2024-04-26T19:59:45.280606+0000","last_became_peered":"2024-04-26T19:59:45.280606+0000","last_unstale":"2024-04-26T19:59:47.294034+0000","last_undegraded":"2024-04-26T19:59:47.294034+0000","last_fullsized":"2024-04-26T19:59:47.294034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464007+0000","last_change":"2024-04-26T19:59:46.296268+0000","last_active":"2024-04-26T19:59:47.464007+0000","last_peered":"2024-04-26T19:59:47.464007+0000","last_clean":"2024-04-26T19:59:47.464007+0000","last_became_active":"2024-04-26T19:59:46.294912+0000","last_became_peered":"2024-04-26T19:59:46.294912+0000","last_unstale":"2024-04-26T19:59:47.464007+0000","last_undegraded":"2024-04-26T19:59:47.464007+0000","last_fullsized":"2024-04-26T19:59:47.464007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469551+0000","last_change":"2024-04-26T19:59:45.299596+0000","last_active":"2024-04-26T19:59:47.469551+0000","last_peered":"2024-04-26T19:59:47.469551+0000","last_clean":"2024-04-26T19:59:47.469551+0000","last_became_active":"2024-04-26T19:59:45.299284+0000","last_became_peered":"2024-04-26T19:59:45.299284+0000","last_unstale":"2024-04-26T19:59:47.469551+0000","last_undegraded":"2024-04-26T19:59:47.469551+0000","last_fullsized":"2024-04-26T19:59:47.469551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465783+0000","last_change":"2024-04-26T19:59:46.294465+0000","last_active":"2024-04-26T19:59:47.465783+0000","last_peered":"2024-04-26T19:59:47.465783+0000","last_clean":"2024-04-26T19:59:47.465783+0000","last_became_active":"2024-04-26T19:59:46.294138+0000","last_became_peered":"2024-04-26T19:59:46.294138+0000","last_unstale":"2024-04-26T19:59:47.465783+0000","last_undegraded":"2024-04-26T19:59:47.465783+0000","last_fullsized":"2024-04-26T19:59:47.465783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467698+0000","last_change":"2024-04-26T19:59:46.304096+0000","last_active":"2024-04-26T19:59:47.467698+0000","last_peered":"2024-04-26T19:59:47.467698+0000","last_clean":"2024-04-26T19:59:47.467698+0000","last_became_active":"2024-04-26T19:59:46.299682+0000","last_became_peered":"2024-04-26T19:59:46.299682+0000","last_unstale":"2024-04-26T19:59:47.467698+0000","last_undegraded":"2024-04-26T19:59:47.467698+0000","last_fullsized":"2024-04-26T19:59:47.467698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467128+0000","last_change":"2024-04-26T19:59:45.302197+0000","last_active":"2024-04-26T19:59:47.467128+0000","last_peered":"2024-04-26T19:59:47.467128+0000","last_clean":"2024-04-26T19:59:47.467128+0000","last_became_active":"2024-04-26T19:59:45.301818+0000","last_became_peered":"2024-04-26T19:59:45.301818+0000","last_unstale":"2024-04-26T19:59:47.467128+0000","last_undegraded":"2024-04-26T19:59:47.467128+0000","last_fullsized":"2024-04-26T19:59:47.467128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:43.563741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466068+0000","last_change":"2024-04-26T19:59:46.309844+0000","last_active":"2024-04-26T19:59:47.466068+0000","last_peered":"2024-04-26T19:59:47.466068+0000","last_clean":"2024-04-26T19:59:47.466068+0000","last_became_active":"2024-04-26T19:59:46.309519+0000","last_became_peered":"2024-04-26T19:59:46.309519+0000","last_unstale":"2024-04-26T19:59:47.466068+0000","last_undegraded":"2024-04-26T19:59:47.466068+0000","last_fullsized":"2024-04-26T19:59:47.466068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467758+0000","last_change":"2024-04-26T19:59:45.301972+0000","last_active":"2024-04-26T19:59:47.467758+0000","last_peered":"2024-04-26T19:59:47.467758+0000","last_clean":"2024-04-26T19:59:47.467758+0000","last_became_active":"2024-04-26T19:59:45.301689+0000","last_became_peered":"2024-04-26T19:59:45.301689+0000","last_unstale":"2024-04-26T19:59:47.467758+0000","last_undegraded":"2024-04-26T19:59:47.467758+0000","last_fullsized":"2024-04-26T19:59:47.467758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463870+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.463870+0000","last_peered":"2024-04-26T19:59:47.463870+0000","last_clean":"2024-04-26T19:59:47.463870+0000","last_became_active":"2024-04-26T19:59:46.290172+0000","last_became_peered":"2024-04-26T19:59:46.290172+0000","last_unstale":"2024-04-26T19:59:47.463870+0000","last_undegraded":"2024-04-26T19:59:47.463870+0000","last_fullsized":"2024-04-26T19:59:47.463870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464753+0000","last_change":"2024-04-26T19:59:45.287597+0000","last_active":"2024-04-26T19:59:47.464753+0000","last_peered":"2024-04-26T19:59:47.464753+0000","last_clean":"2024-04-26T19:59:47.464753+0000","last_became_active":"2024-04-26T19:59:45.287273+0000","last_became_peered":"2024-04-26T19:59:45.287273+0000","last_unstale":"2024-04-26T19:59:47.464753+0000","last_undegraded":"2024-04-26T19:59:47.464753+0000","last_fullsized":"2024-04-26T19:59:47.464753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463701+0000","last_change":"2024-04-26T19:59:46.301596+0000","last_active":"2024-04-26T19:59:47.463701+0000","last_peered":"2024-04-26T19:59:47.463701+0000","last_clean":"2024-04-26T19:59:47.463701+0000","last_became_active":"2024-04-26T19:59:46.292219+0000","last_became_peered":"2024-04-26T19:59:46.292219+0000","last_unstale":"2024-04-26T19:59:47.463701+0000","last_undegraded":"2024-04-26T19:59:47.463701+0000","last_fullsized":"2024-04-26T19:59:47.463701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464191+0000","last_change":"2024-04-26T19:59:45.293237+0000","last_active":"2024-04-26T19:59:47.464191+0000","last_peered":"2024-04-26T19:59:47.464191+0000","last_clean":"2024-04-26T19:59:47.464191+0000","last_became_active":"2024-04-26T19:59:45.292699+0000","last_became_peered":"2024-04-26T19:59:45.292699+0000","last_unstale":"2024-04-26T19:59:47.464191+0000","last_undegraded":"2024-04-26T19:59:47.464191+0000","last_fullsized":"2024-04-26T19:59:47.464191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:59:57.121 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:59:57.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.355+0000 7fc119600640 1 Processor -- start 2024-04-26T19:59:57.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.355+0000 7fc119600640 1 -- start start 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc119600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 0x7fc1140add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc119600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1140af9a0 con 0x7fc11415cdf0 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc119600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1140afc80 con 0x7fc11406ce80 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc119600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc1140ae2c0 con 0x7fc1140cd650 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc112a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc11415cdf0 0x7fc11406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52806/0 (socket says 172.21.15.88:52806) 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc112a00640 1 -- 172.21.15.88:0/4116786032 learned_addr learned my addr 172.21.15.88:0/4116786032 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.357+0000 7fc113400640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 0x7fc1140add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356167762 0 0) 0x7fc1140af9a0 con 0x7fc11415cdf0 2024-04-26T19:59:57.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0f8003520 con 0x7fc11415cdf0 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1285006386 0 0) 0x7fc0f8003520 con 0x7fc11415cdf0 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 msgr2=0x7fc1140add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 0x7fc1140add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 >> v1:172.21.15.132:6789/0 conn(0x7fc11406ce80 legacy=0x7fc1140b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1140aff60 con 0x7fc11415cdf0 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fc104001670 con 0x7fc11415cdf0 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 >> v1:172.21.15.88:6789/0 conn(0x7fc11415cdf0 legacy=0x7fc11406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 0x7fc0f80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0f8003f60 0x7fc0f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8004d40 0x7fc0f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f800ba70 con 0x7fc0f8003f60 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f800bc20 con 0x7fc0f8004d40 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f800be20 con 0x7fc1140cd650 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.358+0000 7fc10be00640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0f8003f60 0x7fc0f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.359+0000 7fc10be00640 1 -- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 msgr2=0x7fc0f80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.359+0000 7fc10be00640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1140cd650 0x7fc0f80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.359+0000 7fc10be00640 1 -- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8004d40 msgr2=0x7fc0f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.359+0000 7fc10be00640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8004d40 0x7fc0f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.359+0000 7fc10be00640 1 -- 172.21.15.88:0/4116786032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f8003520 con 0x7fc0f8003f60 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.360+0000 7fc10be00640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0f8003f60 0x7fc0f8004800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001750 tx=0x7fc0f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757b12cbf03393b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.360+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc001370 con 0x7fc0f8003f60 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.360+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0052d0 con 0x7fc0f8003f60 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.360+0000 7fc112000640 1 -- 172.21.15.88:0/4116786032 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc02abe0 con 0x7fc0f8003f60 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 -- 172.21.15.88:0/4116786032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0f8003f60 msgr2=0x7fc0f8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 --2- 172.21.15.88:0/4116786032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc0f8003f60 0x7fc0f8004800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc001750 tx=0x7fc0f8005830 comp rx=0 tx=0).stop 2024-04-26T19:59:57.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc113400640 1 -- 172.21.15.88:0/4116786032 reap_dead start 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 -- 172.21.15.88:0/4116786032 shutdown_connections 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 -- 172.21.15.88:0/4116786032 >> 172.21.15.88:0/4116786032 conn(0x7fc1140b2d50 msgr2=0x7fc1140b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 -- 172.21.15.88:0/4116786032 shutdown_connections 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.361+0000 7fc119600640 1 -- 172.21.15.88:0/4116786032 wait complete. 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.363+0000 7fc119600640 1 Processor -- start 2024-04-26T19:59:57.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.363+0000 7fc119600640 1 -- start start 2024-04-26T19:59:57.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 0x7fc114156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 0x7fc11415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc11415d1d0 con 0x7fc11415ff30 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc1140b05b0 con 0x7fc0f8003f60 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc119600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc11415d350 con 0x7fc11409ba50 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45298/0 (socket says 172.21.15.88:45298) 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 -- 172.21.15.88:0/3974039629 learned_addr learned my addr 172.21.15.88:0/3974039629 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc112a00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 0x7fc114156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc10be00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 0x7fc11415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 -- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 msgr2=0x7fc11415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 0x7fc11415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 -- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 msgr2=0x7fc114156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 0x7fc114156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc113400640 1 -- 172.21.15.88:0/3974039629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc114156c90 con 0x7fc11415ff30 2024-04-26T19:59:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc112a00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 0x7fc114156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.364+0000 7fc10be00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 0x7fc11415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc113400640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1140d0380 tx=0x7fc10c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c018020 con 0x7fc11415ff30 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c00bb10 con 0x7fc11415ff30 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c014400 con 0x7fc11415ff30 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11409e3c0 con 0x7fc11415ff30 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.365+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141574a0 con 0x7fc11415ff30 2024-04-26T19:59:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.366+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11406ca70 con 0x7fc11415ff30 2024-04-26T19:59:57.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.368+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc10c005d10 con 0x7fc11415ff30 2024-04-26T19:59:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.369+0000 7fc10aa00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.369+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc10c08c210 con 0x7fc11415ff30 2024-04-26T19:59:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.369+0000 7fc112a00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.370+0000 7fc112a00640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc104007ab0 tx=0x7fc104009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.374+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc11406ca70 con 0x7fc11415ff30 2024-04-26T19:59:57.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.567+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc114157290 con 0x7fc0f006c8f0 2024-04-26T19:59:57.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.577+0000 7fc10aa00640 1 -- 172.21.15.88:0/3974039629 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fc114157290 con 0x7fc0f006c8f0 2024-04-26T19:59:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:57.583 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T19:59:57.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 msgr2=0x7fc0f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 0x7fc0f006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc104007ab0 tx=0x7fc104009040 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 msgr2=0x7fc1141f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1140d0380 tx=0x7fc10c00b480 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 shutdown_connections 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11415ff30 0x7fc1141f2d70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0f006c8f0 0x7fc0f006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc11409ba50 0x7fc11415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 --2- 172.21.15.88:0/3974039629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc0f8003f60 0x7fc114156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 >> 172.21.15.88:0/3974039629 conn(0x7fc1140b2d50 msgr2=0x7fc1140d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 shutdown_connections 2024-04-26T19:59:57.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.582+0000 7fc119600640 1 -- 172.21.15.88:0/3974039629 wait complete. 2024-04-26T19:59:57.602 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T19:59:56.943938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003330"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465394+0000","last_change":"2024-04-26T19:59:46.299176+0000","last_active":"2024-04-26T19:59:47.465394+0000","last_peered":"2024-04-26T19:59:47.465394+0000","last_clean":"2024-04-26T19:59:47.465394+0000","last_became_active":"2024-04-26T19:59:46.298473+0000","last_became_peered":"2024-04-26T19:59:46.298473+0000","last_unstale":"2024-04-26T19:59:47.465394+0000","last_undegraded":"2024-04-26T19:59:47.465394+0000","last_fullsized":"2024-04-26T19:59:47.465394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294427+0000","last_change":"2024-04-26T19:59:46.279518+0000","last_active":"2024-04-26T19:59:47.294427+0000","last_peered":"2024-04-26T19:59:47.294427+0000","last_clean":"2024-04-26T19:59:47.294427+0000","last_became_active":"2024-04-26T19:59:46.279235+0000","last_became_peered":"2024-04-26T19:59:46.279235+0000","last_unstale":"2024-04-26T19:59:47.294427+0000","last_undegraded":"2024-04-26T19:59:47.294427+0000","last_fullsized":"2024-04-26T19:59:47.294427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465545+0000","last_change":"2024-04-26T19:59:46.293998+0000","last_active":"2024-04-26T19:59:47.465545+0000","last_peered":"2024-04-26T19:59:47.465545+0000","last_clean":"2024-04-26T19:59:47.465545+0000","last_became_active":"2024-04-26T19:59:46.293698+0000","last_became_peered":"2024-04-26T19:59:46.293698+0000","last_unstale":"2024-04-26T19:59:47.465545+0000","last_undegraded":"2024-04-26T19:59:47.465545+0000","last_fullsized":"2024-04-26T19:59:47.465545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465774+0000","last_change":"2024-04-26T19:59:46.295413+0000","last_active":"2024-04-26T19:59:47.465774+0000","last_peered":"2024-04-26T19:59:47.465774+0000","last_clean":"2024-04-26T19:59:47.465774+0000","last_became_active":"2024-04-26T19:59:46.294973+0000","last_became_peered":"2024-04-26T19:59:46.294973+0000","last_unstale":"2024-04-26T19:59:47.465774+0000","last_undegraded":"2024-04-26T19:59:47.465774+0000","last_fullsized":"2024-04-26T19:59:47.465774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469290+0000","last_change":"2024-04-26T19:59:46.299569+0000","last_active":"2024-04-26T19:59:47.469290+0000","last_peered":"2024-04-26T19:59:47.469290+0000","last_clean":"2024-04-26T19:59:47.469290+0000","last_became_active":"2024-04-26T19:59:46.299229+0000","last_became_peered":"2024-04-26T19:59:46.299229+0000","last_unstale":"2024-04-26T19:59:47.469290+0000","last_undegraded":"2024-04-26T19:59:47.469290+0000","last_fullsized":"2024-04-26T19:59:47.469290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464401+0000","last_change":"2024-04-26T19:59:46.287705+0000","last_active":"2024-04-26T19:59:47.464401+0000","last_peered":"2024-04-26T19:59:47.464401+0000","last_clean":"2024-04-26T19:59:47.464401+0000","last_became_active":"2024-04-26T19:59:46.287398+0000","last_became_peered":"2024-04-26T19:59:46.287398+0000","last_unstale":"2024-04-26T19:59:47.464401+0000","last_undegraded":"2024-04-26T19:59:47.464401+0000","last_fullsized":"2024-04-26T19:59:47.464401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466440+0000","last_change":"2024-04-26T19:59:46.304005+0000","last_active":"2024-04-26T19:59:47.466440+0000","last_peered":"2024-04-26T19:59:47.466440+0000","last_clean":"2024-04-26T19:59:47.466440+0000","last_became_active":"2024-04-26T19:59:46.303696+0000","last_became_peered":"2024-04-26T19:59:46.303696+0000","last_unstale":"2024-04-26T19:59:47.466440+0000","last_undegraded":"2024-04-26T19:59:47.466440+0000","last_fullsized":"2024-04-26T19:59:47.466440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465106+0000","last_change":"2024-04-26T19:59:46.294993+0000","last_active":"2024-04-26T19:59:47.465106+0000","last_peered":"2024-04-26T19:59:47.465106+0000","last_clean":"2024-04-26T19:59:47.465106+0000","last_became_active":"2024-04-26T19:59:46.294405+0000","last_became_peered":"2024-04-26T19:59:46.294405+0000","last_unstale":"2024-04-26T19:59:47.465106+0000","last_undegraded":"2024-04-26T19:59:47.465106+0000","last_fullsized":"2024-04-26T19:59:47.465106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465470+0000","last_change":"2024-04-26T19:59:46.311322+0000","last_active":"2024-04-26T19:59:47.465470+0000","last_peered":"2024-04-26T19:59:47.465470+0000","last_clean":"2024-04-26T19:59:47.465470+0000","last_became_active":"2024-04-26T19:59:46.308499+0000","last_became_peered":"2024-04-26T19:59:46.308499+0000","last_unstale":"2024-04-26T19:59:47.465470+0000","last_undegraded":"2024-04-26T19:59:47.465470+0000","last_fullsized":"2024-04-26T19:59:47.465470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:54.493920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469431+0000","last_change":"2024-04-26T19:59:46.292325+0000","last_active":"2024-04-26T19:59:47.469431+0000","last_peered":"2024-04-26T19:59:47.469431+0000","last_clean":"2024-04-26T19:59:47.469431+0000","last_became_active":"2024-04-26T19:59:46.288719+0000","last_became_peered":"2024-04-26T19:59:46.288719+0000","last_unstale":"2024-04-26T19:59:47.469431+0000","last_undegraded":"2024-04-26T19:59:47.469431+0000","last_fullsized":"2024-04-26T19:59:47.469431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465975+0000","last_change":"2024-04-26T19:59:46.290820+0000","last_active":"2024-04-26T19:59:47.465975+0000","last_peered":"2024-04-26T19:59:47.465975+0000","last_clean":"2024-04-26T19:59:47.465975+0000","last_became_active":"2024-04-26T19:59:46.289935+0000","last_became_peered":"2024-04-26T19:59:46.289935+0000","last_unstale":"2024-04-26T19:59:47.465975+0000","last_undegraded":"2024-04-26T19:59:47.465975+0000","last_fullsized":"2024-04-26T19:59:47.465975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298228+0000","last_change":"2024-04-26T19:59:46.307273+0000","last_active":"2024-04-26T19:59:47.298228+0000","last_peered":"2024-04-26T19:59:47.298228+0000","last_clean":"2024-04-26T19:59:47.298228+0000","last_became_active":"2024-04-26T19:59:46.306426+0000","last_became_peered":"2024-04-26T19:59:46.306426+0000","last_unstale":"2024-04-26T19:59:47.298228+0000","last_undegraded":"2024-04-26T19:59:47.298228+0000","last_fullsized":"2024-04-26T19:59:47.298228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:35.418105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463934+0000","last_change":"2024-04-26T19:59:46.304263+0000","last_active":"2024-04-26T19:59:47.463934+0000","last_peered":"2024-04-26T19:59:47.463934+0000","last_clean":"2024-04-26T19:59:47.463934+0000","last_became_active":"2024-04-26T19:59:46.303977+0000","last_became_peered":"2024-04-26T19:59:46.303977+0000","last_unstale":"2024-04-26T19:59:47.463934+0000","last_undegraded":"2024-04-26T19:59:47.463934+0000","last_fullsized":"2024-04-26T19:59:47.463934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464499+0000","last_change":"2024-04-26T19:59:46.280420+0000","last_active":"2024-04-26T19:59:47.464499+0000","last_peered":"2024-04-26T19:59:47.464499+0000","last_clean":"2024-04-26T19:59:47.464499+0000","last_became_active":"2024-04-26T19:59:46.279398+0000","last_became_peered":"2024-04-26T19:59:46.279398+0000","last_unstale":"2024-04-26T19:59:47.464499+0000","last_undegraded":"2024-04-26T19:59:47.464499+0000","last_fullsized":"2024-04-26T19:59:47.464499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293703+0000","last_change":"2024-04-26T19:59:46.310086+0000","last_active":"2024-04-26T19:59:47.293703+0000","last_peered":"2024-04-26T19:59:47.293703+0000","last_clean":"2024-04-26T19:59:47.293703+0000","last_became_active":"2024-04-26T19:59:46.309760+0000","last_became_peered":"2024-04-26T19:59:46.309760+0000","last_unstale":"2024-04-26T19:59:47.293703+0000","last_undegraded":"2024-04-26T19:59:47.293703+0000","last_fullsized":"2024-04-26T19:59:47.293703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464282+0000","last_change":"2024-04-26T19:59:46.290194+0000","last_active":"2024-04-26T19:59:47.464282+0000","last_peered":"2024-04-26T19:59:47.464282+0000","last_clean":"2024-04-26T19:59:47.464282+0000","last_became_active":"2024-04-26T19:59:46.289473+0000","last_became_peered":"2024-04-26T19:59:46.289473+0000","last_unstale":"2024-04-26T19:59:47.464282+0000","last_undegraded":"2024-04-26T19:59:47.464282+0000","last_fullsized":"2024-04-26T19:59:47.464282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468794+0000","last_change":"2024-04-26T19:59:46.312802+0000","last_active":"2024-04-26T19:59:47.468794+0000","last_peered":"2024-04-26T19:59:47.468794+0000","last_clean":"2024-04-26T19:59:47.468794+0000","last_became_active":"2024-04-26T19:59:46.312497+0000","last_became_peered":"2024-04-26T19:59:46.312497+0000","last_unstale":"2024-04-26T19:59:47.468794+0000","last_undegraded":"2024-04-26T19:59:47.468794+0000","last_fullsized":"2024-04-26T19:59:47.468794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297946+0000","last_change":"2024-04-26T19:59:46.290353+0000","last_active":"2024-04-26T19:59:47.297946+0000","last_peered":"2024-04-26T19:59:47.297946+0000","last_clean":"2024-04-26T19:59:47.297946+0000","last_became_active":"2024-04-26T19:59:46.289992+0000","last_became_peered":"2024-04-26T19:59:46.289992+0000","last_unstale":"2024-04-26T19:59:47.297946+0000","last_undegraded":"2024-04-26T19:59:47.297946+0000","last_fullsized":"2024-04-26T19:59:47.297946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468211+0000","last_change":"2024-04-26T19:59:46.302429+0000","last_active":"2024-04-26T19:59:47.468211+0000","last_peered":"2024-04-26T19:59:47.468211+0000","last_clean":"2024-04-26T19:59:47.468211+0000","last_became_active":"2024-04-26T19:59:46.302123+0000","last_became_peered":"2024-04-26T19:59:46.302123+0000","last_unstale":"2024-04-26T19:59:47.468211+0000","last_undegraded":"2024-04-26T19:59:47.468211+0000","last_fullsized":"2024-04-26T19:59:47.468211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468516+0000","last_change":"2024-04-26T19:59:46.300571+0000","last_active":"2024-04-26T19:59:47.468516+0000","last_peered":"2024-04-26T19:59:47.468516+0000","last_clean":"2024-04-26T19:59:47.468516+0000","last_became_active":"2024-04-26T19:59:46.300247+0000","last_became_peered":"2024-04-26T19:59:46.300247+0000","last_unstale":"2024-04-26T19:59:47.468516+0000","last_undegraded":"2024-04-26T19:59:47.468516+0000","last_fullsized":"2024-04-26T19:59:47.468516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464971+0000","last_change":"2024-04-26T19:59:46.300279+0000","last_active":"2024-04-26T19:59:47.464971+0000","last_peered":"2024-04-26T19:59:47.464971+0000","last_clean":"2024-04-26T19:59:47.464971+0000","last_became_active":"2024-04-26T19:59:46.300053+0000","last_became_peered":"2024-04-26T19:59:46.300053+0000","last_unstale":"2024-04-26T19:59:47.464971+0000","last_undegraded":"2024-04-26T19:59:47.464971+0000","last_fullsized":"2024-04-26T19:59:47.464971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466037+0000","last_change":"2024-04-26T19:59:46.305198+0000","last_active":"2024-04-26T19:59:47.466037+0000","last_peered":"2024-04-26T19:59:47.466037+0000","last_clean":"2024-04-26T19:59:47.466037+0000","last_became_active":"2024-04-26T19:59:46.303321+0000","last_became_peered":"2024-04-26T19:59:46.303321+0000","last_unstale":"2024-04-26T19:59:47.466037+0000","last_undegraded":"2024-04-26T19:59:47.466037+0000","last_fullsized":"2024-04-26T19:59:47.466037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463271+0000","last_change":"2024-04-26T19:59:46.313226+0000","last_active":"2024-04-26T19:59:47.463271+0000","last_peered":"2024-04-26T19:59:47.463271+0000","last_clean":"2024-04-26T19:59:47.463271+0000","last_became_active":"2024-04-26T19:59:46.312933+0000","last_became_peered":"2024-04-26T19:59:46.312933+0000","last_unstale":"2024-04-26T19:59:47.463271+0000","last_undegraded":"2024-04-26T19:59:47.463271+0000","last_fullsized":"2024-04-26T19:59:47.463271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464413+0000","last_change":"2024-04-26T19:59:46.296413+0000","last_active":"2024-04-26T19:59:47.464413+0000","last_peered":"2024-04-26T19:59:47.464413+0000","last_clean":"2024-04-26T19:59:47.464413+0000","last_became_active":"2024-04-26T19:59:46.295760+0000","last_became_peered":"2024-04-26T19:59:46.295760+0000","last_unstale":"2024-04-26T19:59:47.464413+0000","last_undegraded":"2024-04-26T19:59:47.464413+0000","last_fullsized":"2024-04-26T19:59:47.464413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465037+0000","last_change":"2024-04-26T19:59:46.314022+0000","last_active":"2024-04-26T19:59:47.465037+0000","last_peered":"2024-04-26T19:59:47.465037+0000","last_clean":"2024-04-26T19:59:47.465037+0000","last_became_active":"2024-04-26T19:59:46.313768+0000","last_became_peered":"2024-04-26T19:59:46.313768+0000","last_unstale":"2024-04-26T19:59:47.465037+0000","last_undegraded":"2024-04-26T19:59:47.465037+0000","last_fullsized":"2024-04-26T19:59:47.465037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469629+0000","last_change":"2024-04-26T19:59:46.299335+0000","last_active":"2024-04-26T19:59:47.469629+0000","last_peered":"2024-04-26T19:59:47.469629+0000","last_clean":"2024-04-26T19:59:47.469629+0000","last_became_active":"2024-04-26T19:59:46.299048+0000","last_became_peered":"2024-04-26T19:59:46.299048+0000","last_unstale":"2024-04-26T19:59:47.469629+0000","last_undegraded":"2024-04-26T19:59:47.469629+0000","last_fullsized":"2024-04-26T19:59:47.469629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294167+0000","last_change":"2024-04-26T19:59:46.297650+0000","last_active":"2024-04-26T19:59:47.294167+0000","last_peered":"2024-04-26T19:59:47.294167+0000","last_clean":"2024-04-26T19:59:47.294167+0000","last_became_active":"2024-04-26T19:59:46.297368+0000","last_became_peered":"2024-04-26T19:59:46.297368+0000","last_unstale":"2024-04-26T19:59:47.294167+0000","last_undegraded":"2024-04-26T19:59:47.294167+0000","last_fullsized":"2024-04-26T19:59:47.294167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464415+0000","last_change":"2024-04-26T19:59:46.313265+0000","last_active":"2024-04-26T19:59:47.464415+0000","last_peered":"2024-04-26T19:59:47.464415+0000","last_clean":"2024-04-26T19:59:47.464415+0000","last_became_active":"2024-04-26T19:59:46.313035+0000","last_became_peered":"2024-04-26T19:59:46.313035+0000","last_unstale":"2024-04-26T19:59:47.464415+0000","last_undegraded":"2024-04-26T19:59:47.464415+0000","last_fullsized":"2024-04-26T19:59:47.464415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465117+0000","last_change":"2024-04-26T19:59:46.294814+0000","last_active":"2024-04-26T19:59:47.465117+0000","last_peered":"2024-04-26T19:59:47.465117+0000","last_clean":"2024-04-26T19:59:47.465117+0000","last_became_active":"2024-04-26T19:59:46.294277+0000","last_became_peered":"2024-04-26T19:59:46.294277+0000","last_unstale":"2024-04-26T19:59:47.465117+0000","last_undegraded":"2024-04-26T19:59:47.465117+0000","last_fullsized":"2024-04-26T19:59:47.465117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466341+0000","last_change":"2024-04-26T19:59:46.297299+0000","last_active":"2024-04-26T19:59:47.466341+0000","last_peered":"2024-04-26T19:59:47.466341+0000","last_clean":"2024-04-26T19:59:47.466341+0000","last_became_active":"2024-04-26T19:59:46.296375+0000","last_became_peered":"2024-04-26T19:59:46.296375+0000","last_unstale":"2024-04-26T19:59:47.466341+0000","last_undegraded":"2024-04-26T19:59:47.466341+0000","last_fullsized":"2024-04-26T19:59:47.466341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297656+0000","last_change":"2024-04-26T19:59:46.292257+0000","last_active":"2024-04-26T19:59:47.297656+0000","last_peered":"2024-04-26T19:59:47.297656+0000","last_clean":"2024-04-26T19:59:47.297656+0000","last_became_active":"2024-04-26T19:59:46.291980+0000","last_became_peered":"2024-04-26T19:59:46.291980+0000","last_unstale":"2024-04-26T19:59:47.297656+0000","last_undegraded":"2024-04-26T19:59:47.297656+0000","last_fullsized":"2024-04-26T19:59:47.297656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464925+0000","last_change":"2024-04-26T19:59:46.283803+0000","last_active":"2024-04-26T19:59:47.464925+0000","last_peered":"2024-04-26T19:59:47.464925+0000","last_clean":"2024-04-26T19:59:47.464925+0000","last_became_active":"2024-04-26T19:59:46.283455+0000","last_became_peered":"2024-04-26T19:59:46.283455+0000","last_unstale":"2024-04-26T19:59:47.464925+0000","last_undegraded":"2024-04-26T19:59:47.464925+0000","last_fullsized":"2024-04-26T19:59:47.464925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297641+0000","last_change":"2024-04-26T19:59:46.290324+0000","last_active":"2024-04-26T19:59:47.297641+0000","last_peered":"2024-04-26T19:59:47.297641+0000","last_clean":"2024-04-26T19:59:47.297641+0000","last_became_active":"2024-04-26T19:59:46.289926+0000","last_became_peered":"2024-04-26T19:59:46.289926+0000","last_unstale":"2024-04-26T19:59:47.297641+0000","last_undegraded":"2024-04-26T19:59:47.297641+0000","last_fullsized":"2024-04-26T19:59:47.297641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:56.862425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464081+0000","last_change":"2024-04-26T19:59:45.291916+0000","last_active":"2024-04-26T19:59:47.464081+0000","last_peered":"2024-04-26T19:59:47.464081+0000","last_clean":"2024-04-26T19:59:47.464081+0000","last_became_active":"2024-04-26T19:59:45.291242+0000","last_became_peered":"2024-04-26T19:59:45.291242+0000","last_unstale":"2024-04-26T19:59:47.464081+0000","last_undegraded":"2024-04-26T19:59:47.464081+0000","last_fullsized":"2024-04-26T19:59:47.464081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465265+0000","last_change":"2024-04-26T19:59:46.295081+0000","last_active":"2024-04-26T19:59:47.465265+0000","last_peered":"2024-04-26T19:59:47.465265+0000","last_clean":"2024-04-26T19:59:47.465265+0000","last_became_active":"2024-04-26T19:59:46.294800+0000","last_became_peered":"2024-04-26T19:59:46.294800+0000","last_unstale":"2024-04-26T19:59:47.465265+0000","last_undegraded":"2024-04-26T19:59:47.465265+0000","last_fullsized":"2024-04-26T19:59:47.465265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465946+0000","last_change":"2024-04-26T19:59:45.286181+0000","last_active":"2024-04-26T19:59:47.465946+0000","last_peered":"2024-04-26T19:59:47.465946+0000","last_clean":"2024-04-26T19:59:47.465946+0000","last_became_active":"2024-04-26T19:59:45.285851+0000","last_became_peered":"2024-04-26T19:59:45.285851+0000","last_unstale":"2024-04-26T19:59:47.465946+0000","last_undegraded":"2024-04-26T19:59:47.465946+0000","last_fullsized":"2024-04-26T19:59:47.465946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463328+0000","last_change":"2024-04-26T19:59:45.286697+0000","last_active":"2024-04-26T19:59:47.463328+0000","last_peered":"2024-04-26T19:59:47.463328+0000","last_clean":"2024-04-26T19:59:47.463328+0000","last_became_active":"2024-04-26T19:59:45.286293+0000","last_became_peered":"2024-04-26T19:59:45.286293+0000","last_unstale":"2024-04-26T19:59:47.463328+0000","last_undegraded":"2024-04-26T19:59:47.463328+0000","last_fullsized":"2024-04-26T19:59:47.463328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464589+0000","last_change":"2024-04-26T19:59:46.294713+0000","last_active":"2024-04-26T19:59:47.464589+0000","last_peered":"2024-04-26T19:59:47.464589+0000","last_clean":"2024-04-26T19:59:47.464589+0000","last_became_active":"2024-04-26T19:59:46.294022+0000","last_became_peered":"2024-04-26T19:59:46.294022+0000","last_unstale":"2024-04-26T19:59:47.464589+0000","last_undegraded":"2024-04-26T19:59:47.464589+0000","last_fullsized":"2024-04-26T19:59:47.464589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:26.506557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464432+0000","last_change":"2024-04-26T19:59:45.284101+0000","last_active":"2024-04-26T19:59:47.464432+0000","last_peered":"2024-04-26T19:59:47.464432+0000","last_clean":"2024-04-26T19:59:47.464432+0000","last_became_active":"2024-04-26T19:59:45.283784+0000","last_became_peered":"2024-04-26T19:59:45.283784+0000","last_unstale":"2024-04-26T19:59:47.464432+0000","last_undegraded":"2024-04-26T19:59:47.464432+0000","last_fullsized":"2024-04-26T19:59:47.464432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293526+0000","last_change":"2024-04-26T19:59:46.296659+0000","last_active":"2024-04-26T19:59:47.293526+0000","last_peered":"2024-04-26T19:59:47.293526+0000","last_clean":"2024-04-26T19:59:47.293526+0000","last_became_active":"2024-04-26T19:59:46.296362+0000","last_became_peered":"2024-04-26T19:59:46.296362+0000","last_unstale":"2024-04-26T19:59:47.293526+0000","last_undegraded":"2024-04-26T19:59:47.293526+0000","last_fullsized":"2024-04-26T19:59:47.293526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468741+0000","last_change":"2024-04-26T19:59:45.299230+0000","last_active":"2024-04-26T19:59:47.468741+0000","last_peered":"2024-04-26T19:59:47.468741+0000","last_clean":"2024-04-26T19:59:47.468741+0000","last_became_active":"2024-04-26T19:59:45.298903+0000","last_became_peered":"2024-04-26T19:59:45.298903+0000","last_unstale":"2024-04-26T19:59:47.468741+0000","last_undegraded":"2024-04-26T19:59:47.468741+0000","last_fullsized":"2024-04-26T19:59:47.468741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467717+0000","last_change":"2024-04-26T19:59:46.306993+0000","last_active":"2024-04-26T19:59:47.467717+0000","last_peered":"2024-04-26T19:59:47.467717+0000","last_clean":"2024-04-26T19:59:47.467717+0000","last_became_active":"2024-04-26T19:59:46.305722+0000","last_became_peered":"2024-04-26T19:59:46.305722+0000","last_unstale":"2024-04-26T19:59:47.467717+0000","last_undegraded":"2024-04-26T19:59:47.467717+0000","last_fullsized":"2024-04-26T19:59:47.467717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:26.750421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467742+0000","last_change":"2024-04-26T19:59:45.297752+0000","last_active":"2024-04-26T19:59:47.467742+0000","last_peered":"2024-04-26T19:59:47.467742+0000","last_clean":"2024-04-26T19:59:47.467742+0000","last_became_active":"2024-04-26T19:59:45.297462+0000","last_became_peered":"2024-04-26T19:59:45.297462+0000","last_unstale":"2024-04-26T19:59:47.467742+0000","last_undegraded":"2024-04-26T19:59:47.467742+0000","last_fullsized":"2024-04-26T19:59:47.467742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464194+0000","last_change":"2024-04-26T19:59:46.290536+0000","last_active":"2024-04-26T19:59:47.464194+0000","last_peered":"2024-04-26T19:59:47.464194+0000","last_clean":"2024-04-26T19:59:47.464194+0000","last_became_active":"2024-04-26T19:59:46.289737+0000","last_became_peered":"2024-04-26T19:59:46.289737+0000","last_unstale":"2024-04-26T19:59:47.464194+0000","last_undegraded":"2024-04-26T19:59:47.464194+0000","last_fullsized":"2024-04-26T19:59:47.464194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294232+0000","last_change":"2024-04-26T19:59:45.293462+0000","last_active":"2024-04-26T19:59:47.294232+0000","last_peered":"2024-04-26T19:59:47.294232+0000","last_clean":"2024-04-26T19:59:47.294232+0000","last_became_active":"2024-04-26T19:59:45.292997+0000","last_became_peered":"2024-04-26T19:59:45.292997+0000","last_unstale":"2024-04-26T19:59:47.294232+0000","last_undegraded":"2024-04-26T19:59:47.294232+0000","last_fullsized":"2024-04-26T19:59:47.294232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298199+0000","last_change":"2024-04-26T19:59:46.307369+0000","last_active":"2024-04-26T19:59:47.298199+0000","last_peered":"2024-04-26T19:59:47.298199+0000","last_clean":"2024-04-26T19:59:47.298199+0000","last_became_active":"2024-04-26T19:59:46.306973+0000","last_became_peered":"2024-04-26T19:59:46.306973+0000","last_unstale":"2024-04-26T19:59:47.298199+0000","last_undegraded":"2024-04-26T19:59:47.298199+0000","last_fullsized":"2024-04-26T19:59:47.298199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:41.402397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469706+0000","last_change":"2024-04-26T19:59:45.292264+0000","last_active":"2024-04-26T19:59:47.469706+0000","last_peered":"2024-04-26T19:59:47.469706+0000","last_clean":"2024-04-26T19:59:47.469706+0000","last_became_active":"2024-04-26T19:59:45.290211+0000","last_became_peered":"2024-04-26T19:59:45.290211+0000","last_unstale":"2024-04-26T19:59:47.469706+0000","last_undegraded":"2024-04-26T19:59:47.469706+0000","last_fullsized":"2024-04-26T19:59:47.469706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467202+0000","last_change":"2024-04-26T19:59:46.294340+0000","last_active":"2024-04-26T19:59:47.467202+0000","last_peered":"2024-04-26T19:59:47.467202+0000","last_clean":"2024-04-26T19:59:47.467202+0000","last_became_active":"2024-04-26T19:59:46.294017+0000","last_became_peered":"2024-04-26T19:59:46.294017+0000","last_unstale":"2024-04-26T19:59:47.467202+0000","last_undegraded":"2024-04-26T19:59:47.467202+0000","last_fullsized":"2024-04-26T19:59:47.467202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465254+0000","last_change":"2024-04-26T19:59:45.285166+0000","last_active":"2024-04-26T19:59:47.465254+0000","last_peered":"2024-04-26T19:59:47.465254+0000","last_clean":"2024-04-26T19:59:47.465254+0000","last_became_active":"2024-04-26T19:59:45.284863+0000","last_became_peered":"2024-04-26T19:59:45.284863+0000","last_unstale":"2024-04-26T19:59:47.465254+0000","last_undegraded":"2024-04-26T19:59:47.465254+0000","last_fullsized":"2024-04-26T19:59:47.465254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464651+0000","last_change":"2024-04-26T19:59:46.296940+0000","last_active":"2024-04-26T19:59:47.464651+0000","last_peered":"2024-04-26T19:59:47.464651+0000","last_clean":"2024-04-26T19:59:47.464651+0000","last_became_active":"2024-04-26T19:59:46.296695+0000","last_became_peered":"2024-04-26T19:59:46.296695+0000","last_unstale":"2024-04-26T19:59:47.464651+0000","last_undegraded":"2024-04-26T19:59:47.464651+0000","last_fullsized":"2024-04-26T19:59:47.464651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466255+0000","last_change":"2024-04-26T19:59:45.288319+0000","last_active":"2024-04-26T19:59:47.466255+0000","last_peered":"2024-04-26T19:59:47.466255+0000","last_clean":"2024-04-26T19:59:47.466255+0000","last_became_active":"2024-04-26T19:59:45.287908+0000","last_became_peered":"2024-04-26T19:59:45.287908+0000","last_unstale":"2024-04-26T19:59:47.466255+0000","last_undegraded":"2024-04-26T19:59:47.466255+0000","last_fullsized":"2024-04-26T19:59:47.466255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469394+0000","last_change":"2024-04-26T19:59:46.314258+0000","last_active":"2024-04-26T19:59:47.469394+0000","last_peered":"2024-04-26T19:59:47.469394+0000","last_clean":"2024-04-26T19:59:47.469394+0000","last_became_active":"2024-04-26T19:59:46.313941+0000","last_became_peered":"2024-04-26T19:59:46.313941+0000","last_unstale":"2024-04-26T19:59:47.469394+0000","last_undegraded":"2024-04-26T19:59:47.469394+0000","last_fullsized":"2024-04-26T19:59:47.469394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.441257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466928+0000","last_change":"2024-04-26T19:59:45.292499+0000","last_active":"2024-04-26T19:59:47.466928+0000","last_peered":"2024-04-26T19:59:47.466928+0000","last_clean":"2024-04-26T19:59:47.466928+0000","last_became_active":"2024-04-26T19:59:45.288991+0000","last_became_peered":"2024-04-26T19:59:45.288991+0000","last_unstale":"2024-04-26T19:59:47.466928+0000","last_undegraded":"2024-04-26T19:59:47.466928+0000","last_fullsized":"2024-04-26T19:59:47.466928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465852+0000","last_change":"2024-04-26T19:59:46.283597+0000","last_active":"2024-04-26T19:59:47.465852+0000","last_peered":"2024-04-26T19:59:47.465852+0000","last_clean":"2024-04-26T19:59:47.465852+0000","last_became_active":"2024-04-26T19:59:46.283235+0000","last_became_peered":"2024-04-26T19:59:46.283235+0000","last_unstale":"2024-04-26T19:59:47.465852+0000","last_undegraded":"2024-04-26T19:59:47.465852+0000","last_fullsized":"2024-04-26T19:59:47.465852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:17.414236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297270+0000","last_change":"2024-04-26T19:59:45.276538+0000","last_active":"2024-04-26T19:59:47.297270+0000","last_peered":"2024-04-26T19:59:47.297270+0000","last_clean":"2024-04-26T19:59:47.297270+0000","last_became_active":"2024-04-26T19:59:45.276197+0000","last_became_peered":"2024-04-26T19:59:45.276197+0000","last_unstale":"2024-04-26T19:59:47.297270+0000","last_undegraded":"2024-04-26T19:59:47.297270+0000","last_fullsized":"2024-04-26T19:59:47.297270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466760+0000","last_change":"2024-04-26T19:59:46.314180+0000","last_active":"2024-04-26T19:59:47.466760+0000","last_peered":"2024-04-26T19:59:47.466760+0000","last_clean":"2024-04-26T19:59:47.466760+0000","last_became_active":"2024-04-26T19:59:46.313845+0000","last_became_peered":"2024-04-26T19:59:46.313845+0000","last_unstale":"2024-04-26T19:59:47.466760+0000","last_undegraded":"2024-04-26T19:59:47.466760+0000","last_fullsized":"2024-04-26T19:59:47.466760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:39.294040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.298083+0000","last_change":"2024-04-26T19:59:45.282738+0000","last_active":"2024-04-26T19:59:47.298083+0000","last_peered":"2024-04-26T19:59:47.298083+0000","last_clean":"2024-04-26T19:59:47.298083+0000","last_became_active":"2024-04-26T19:59:45.281228+0000","last_became_peered":"2024-04-26T19:59:45.281228+0000","last_unstale":"2024-04-26T19:59:47.298083+0000","last_undegraded":"2024-04-26T19:59:47.298083+0000","last_fullsized":"2024-04-26T19:59:47.298083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465206+0000","last_change":"2024-04-26T19:59:46.314419+0000","last_active":"2024-04-26T19:59:47.465206+0000","last_peered":"2024-04-26T19:59:47.465206+0000","last_clean":"2024-04-26T19:59:47.465206+0000","last_became_active":"2024-04-26T19:59:46.313946+0000","last_became_peered":"2024-04-26T19:59:46.313946+0000","last_unstale":"2024-04-26T19:59:47.465206+0000","last_undegraded":"2024-04-26T19:59:47.465206+0000","last_fullsized":"2024-04-26T19:59:47.465206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297898+0000","last_change":"2024-04-26T19:59:45.266464+0000","last_active":"2024-04-26T19:59:47.297898+0000","last_peered":"2024-04-26T19:59:47.297898+0000","last_clean":"2024-04-26T19:59:47.297898+0000","last_became_active":"2024-04-26T19:59:45.265937+0000","last_became_peered":"2024-04-26T19:59:45.265937+0000","last_unstale":"2024-04-26T19:59:47.297898+0000","last_undegraded":"2024-04-26T19:59:47.297898+0000","last_fullsized":"2024-04-26T19:59:47.297898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:00:35.753658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465838+0000","last_change":"2024-04-26T19:59:46.314568+0000","last_active":"2024-04-26T19:59:47.465838+0000","last_peered":"2024-04-26T19:59:47.465838+0000","last_clean":"2024-04-26T19:59:47.465838+0000","last_became_active":"2024-04-26T19:59:46.313865+0000","last_became_peered":"2024-04-26T19:59:46.313865+0000","last_unstale":"2024-04-26T19:59:47.465838+0000","last_undegraded":"2024-04-26T19:59:47.465838+0000","last_fullsized":"2024-04-26T19:59:47.465838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294351+0000","last_change":"2024-04-26T19:59:45.281434+0000","last_active":"2024-04-26T19:59:47.294351+0000","last_peered":"2024-04-26T19:59:47.294351+0000","last_clean":"2024-04-26T19:59:47.294351+0000","last_became_active":"2024-04-26T19:59:45.280816+0000","last_became_peered":"2024-04-26T19:59:45.280816+0000","last_unstale":"2024-04-26T19:59:47.294351+0000","last_undegraded":"2024-04-26T19:59:47.294351+0000","last_fullsized":"2024-04-26T19:59:47.294351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466518+0000","last_change":"2024-04-26T19:59:46.295918+0000","last_active":"2024-04-26T19:59:47.466518+0000","last_peered":"2024-04-26T19:59:47.466518+0000","last_clean":"2024-04-26T19:59:47.466518+0000","last_became_active":"2024-04-26T19:59:46.295630+0000","last_became_peered":"2024-04-26T19:59:46.295630+0000","last_unstale":"2024-04-26T19:59:47.466518+0000","last_undegraded":"2024-04-26T19:59:47.466518+0000","last_fullsized":"2024-04-26T19:59:47.466518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:35.219801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297781+0000","last_change":"2024-04-26T19:59:45.288285+0000","last_active":"2024-04-26T19:59:47.297781+0000","last_peered":"2024-04-26T19:59:47.297781+0000","last_clean":"2024-04-26T19:59:47.297781+0000","last_became_active":"2024-04-26T19:59:45.287962+0000","last_became_peered":"2024-04-26T19:59:45.287962+0000","last_unstale":"2024-04-26T19:59:47.297781+0000","last_undegraded":"2024-04-26T19:59:47.297781+0000","last_fullsized":"2024-04-26T19:59:47.297781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:26:07.871484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467597+0000","last_change":"2024-04-26T19:59:46.310806+0000","last_active":"2024-04-26T19:59:47.467597+0000","last_peered":"2024-04-26T19:59:47.467597+0000","last_clean":"2024-04-26T19:59:47.467597+0000","last_became_active":"2024-04-26T19:59:46.310281+0000","last_became_peered":"2024-04-26T19:59:46.310281+0000","last_unstale":"2024-04-26T19:59:47.467597+0000","last_undegraded":"2024-04-26T19:59:47.467597+0000","last_fullsized":"2024-04-26T19:59:47.467597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464724+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T19:59:47.464724+0000","last_peered":"2024-04-26T19:59:47.464724+0000","last_clean":"2024-04-26T19:59:47.464724+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T19:59:47.464724+0000","last_undegraded":"2024-04-26T19:59:47.464724+0000","last_fullsized":"2024-04-26T19:59:47.464724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465285+0000","last_change":"2024-04-26T19:59:45.281392+0000","last_active":"2024-04-26T19:59:47.465285+0000","last_peered":"2024-04-26T19:59:47.465285+0000","last_clean":"2024-04-26T19:59:47.465285+0000","last_became_active":"2024-04-26T19:59:45.281041+0000","last_became_peered":"2024-04-26T19:59:45.281041+0000","last_unstale":"2024-04-26T19:59:47.465285+0000","last_undegraded":"2024-04-26T19:59:47.465285+0000","last_fullsized":"2024-04-26T19:59:47.465285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465458+0000","last_change":"2024-04-26T19:59:46.294932+0000","last_active":"2024-04-26T19:59:47.465458+0000","last_peered":"2024-04-26T19:59:47.465458+0000","last_clean":"2024-04-26T19:59:47.465458+0000","last_became_active":"2024-04-26T19:59:46.294670+0000","last_became_peered":"2024-04-26T19:59:46.294670+0000","last_unstale":"2024-04-26T19:59:47.465458+0000","last_undegraded":"2024-04-26T19:59:47.465458+0000","last_fullsized":"2024-04-26T19:59:47.465458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464419+0000","last_change":"2024-04-26T19:59:45.299670+0000","last_active":"2024-04-26T19:59:47.464419+0000","last_peered":"2024-04-26T19:59:47.464419+0000","last_clean":"2024-04-26T19:59:47.464419+0000","last_became_active":"2024-04-26T19:59:45.299344+0000","last_became_peered":"2024-04-26T19:59:45.299344+0000","last_unstale":"2024-04-26T19:59:47.464419+0000","last_undegraded":"2024-04-26T19:59:47.464419+0000","last_fullsized":"2024-04-26T19:59:47.464419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:17.563470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465122+0000","last_change":"2024-04-26T19:59:46.300383+0000","last_active":"2024-04-26T19:59:47.465122+0000","last_peered":"2024-04-26T19:59:47.465122+0000","last_clean":"2024-04-26T19:59:47.465122+0000","last_became_active":"2024-04-26T19:59:46.299990+0000","last_became_peered":"2024-04-26T19:59:46.299990+0000","last_unstale":"2024-04-26T19:59:47.465122+0000","last_undegraded":"2024-04-26T19:59:47.465122+0000","last_fullsized":"2024-04-26T19:59:47.465122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297425+0000","last_change":"2024-04-26T19:59:45.270294+0000","last_active":"2024-04-26T19:59:47.297425+0000","last_peered":"2024-04-26T19:59:47.297425+0000","last_clean":"2024-04-26T19:59:47.297425+0000","last_became_active":"2024-04-26T19:59:45.269847+0000","last_became_peered":"2024-04-26T19:59:45.269847+0000","last_unstale":"2024-04-26T19:59:47.297425+0000","last_undegraded":"2024-04-26T19:59:47.297425+0000","last_fullsized":"2024-04-26T19:59:47.297425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:14:08.362585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294086+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.294086+0000","last_peered":"2024-04-26T19:59:47.294086+0000","last_clean":"2024-04-26T19:59:47.294086+0000","last_became_active":"2024-04-26T19:59:46.290208+0000","last_became_peered":"2024-04-26T19:59:46.290208+0000","last_unstale":"2024-04-26T19:59:47.294086+0000","last_undegraded":"2024-04-26T19:59:47.294086+0000","last_fullsized":"2024-04-26T19:59:47.294086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:27:12.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466469+0000","last_change":"2024-04-26T19:59:45.292265+0000","last_active":"2024-04-26T19:59:47.466469+0000","last_peered":"2024-04-26T19:59:47.466469+0000","last_clean":"2024-04-26T19:59:47.466469+0000","last_became_active":"2024-04-26T19:59:45.291785+0000","last_became_peered":"2024-04-26T19:59:45.291785+0000","last_unstale":"2024-04-26T19:59:47.466469+0000","last_undegraded":"2024-04-26T19:59:47.466469+0000","last_fullsized":"2024-04-26T19:59:47.466469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297995+0000","last_change":"2024-04-26T19:59:46.294373+0000","last_active":"2024-04-26T19:59:47.297995+0000","last_peered":"2024-04-26T19:59:47.297995+0000","last_clean":"2024-04-26T19:59:47.297995+0000","last_became_active":"2024-04-26T19:59:46.292939+0000","last_became_peered":"2024-04-26T19:59:46.292939+0000","last_unstale":"2024-04-26T19:59:47.297995+0000","last_undegraded":"2024-04-26T19:59:47.297995+0000","last_fullsized":"2024-04-26T19:59:47.297995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:45.057739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294407+0000","last_change":"2024-04-26T19:59:45.286062+0000","last_active":"2024-04-26T19:59:47.294407+0000","last_peered":"2024-04-26T19:59:47.294407+0000","last_clean":"2024-04-26T19:59:47.294407+0000","last_became_active":"2024-04-26T19:59:45.282680+0000","last_became_peered":"2024-04-26T19:59:45.282680+0000","last_unstale":"2024-04-26T19:59:47.294407+0000","last_undegraded":"2024-04-26T19:59:47.294407+0000","last_fullsized":"2024-04-26T19:59:47.294407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467709+0000","last_change":"2024-04-26T19:59:46.303855+0000","last_active":"2024-04-26T19:59:47.467709+0000","last_peered":"2024-04-26T19:59:47.467709+0000","last_clean":"2024-04-26T19:59:47.467709+0000","last_became_active":"2024-04-26T19:59:46.298323+0000","last_became_peered":"2024-04-26T19:59:46.298323+0000","last_unstale":"2024-04-26T19:59:47.467709+0000","last_undegraded":"2024-04-26T19:59:47.467709+0000","last_fullsized":"2024-04-26T19:59:47.467709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:43.494471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468317+0000","last_change":"2024-04-26T19:59:45.302616+0000","last_active":"2024-04-26T19:59:47.468317+0000","last_peered":"2024-04-26T19:59:47.468317+0000","last_clean":"2024-04-26T19:59:47.468317+0000","last_became_active":"2024-04-26T19:59:45.302164+0000","last_became_peered":"2024-04-26T19:59:45.302164+0000","last_unstale":"2024-04-26T19:59:47.468317+0000","last_undegraded":"2024-04-26T19:59:47.468317+0000","last_fullsized":"2024-04-26T19:59:47.468317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294273+0000","last_change":"2024-04-26T19:59:46.290753+0000","last_active":"2024-04-26T19:59:47.294273+0000","last_peered":"2024-04-26T19:59:47.294273+0000","last_clean":"2024-04-26T19:59:47.294273+0000","last_became_active":"2024-04-26T19:59:46.290477+0000","last_became_peered":"2024-04-26T19:59:46.290477+0000","last_unstale":"2024-04-26T19:59:47.294273+0000","last_undegraded":"2024-04-26T19:59:47.294273+0000","last_fullsized":"2024-04-26T19:59:47.294273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:18.371422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294611+0000","last_change":"2024-04-26T19:59:45.297751+0000","last_active":"2024-04-26T19:59:47.294611+0000","last_peered":"2024-04-26T19:59:47.294611+0000","last_clean":"2024-04-26T19:59:47.294611+0000","last_became_active":"2024-04-26T19:59:45.297426+0000","last_became_peered":"2024-04-26T19:59:45.297426+0000","last_unstale":"2024-04-26T19:59:47.294611+0000","last_undegraded":"2024-04-26T19:59:47.294611+0000","last_fullsized":"2024-04-26T19:59:47.294611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465393+0000","last_change":"2024-04-26T19:59:46.298839+0000","last_active":"2024-04-26T19:59:47.465393+0000","last_peered":"2024-04-26T19:59:47.465393+0000","last_clean":"2024-04-26T19:59:47.465393+0000","last_became_active":"2024-04-26T19:59:46.298552+0000","last_became_peered":"2024-04-26T19:59:46.298552+0000","last_unstale":"2024-04-26T19:59:47.465393+0000","last_undegraded":"2024-04-26T19:59:47.465393+0000","last_fullsized":"2024-04-26T19:59:47.465393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:26.141231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.468933+0000","last_change":"2024-04-26T19:59:45.291668+0000","last_active":"2024-04-26T19:59:47.468933+0000","last_peered":"2024-04-26T19:59:47.468933+0000","last_clean":"2024-04-26T19:59:47.468933+0000","last_became_active":"2024-04-26T19:59:45.291333+0000","last_became_peered":"2024-04-26T19:59:45.291333+0000","last_unstale":"2024-04-26T19:59:47.468933+0000","last_undegraded":"2024-04-26T19:59:47.468933+0000","last_fullsized":"2024-04-26T19:59:47.468933+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465754+0000","last_change":"2024-04-26T19:59:46.276488+0000","last_active":"2024-04-26T19:59:47.465754+0000","last_peered":"2024-04-26T19:59:47.465754+0000","last_clean":"2024-04-26T19:59:47.465754+0000","last_became_active":"2024-04-26T19:59:46.274349+0000","last_became_peered":"2024-04-26T19:59:46.274349+0000","last_unstale":"2024-04-26T19:59:47.465754+0000","last_undegraded":"2024-04-26T19:59:47.465754+0000","last_fullsized":"2024-04-26T19:59:47.465754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:32.241894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464100+0000","last_change":"2024-04-26T19:59:45.300210+0000","last_active":"2024-04-26T19:59:47.464100+0000","last_peered":"2024-04-26T19:59:47.464100+0000","last_clean":"2024-04-26T19:59:47.464100+0000","last_became_active":"2024-04-26T19:59:45.299954+0000","last_became_peered":"2024-04-26T19:59:45.299954+0000","last_unstale":"2024-04-26T19:59:47.464100+0000","last_undegraded":"2024-04-26T19:59:47.464100+0000","last_fullsized":"2024-04-26T19:59:47.464100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:55.846792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.297802+0000","last_change":"2024-04-26T19:59:46.299193+0000","last_active":"2024-04-26T19:59:47.297802+0000","last_peered":"2024-04-26T19:59:47.297802+0000","last_clean":"2024-04-26T19:59:47.297802+0000","last_became_active":"2024-04-26T19:59:46.298053+0000","last_became_peered":"2024-04-26T19:59:46.298053+0000","last_unstale":"2024-04-26T19:59:47.297802+0000","last_undegraded":"2024-04-26T19:59:47.297802+0000","last_fullsized":"2024-04-26T19:59:47.297802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:18.579103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.293761+0000","last_change":"2024-04-26T19:59:45.290679+0000","last_active":"2024-04-26T19:59:47.293761+0000","last_peered":"2024-04-26T19:59:47.293761+0000","last_clean":"2024-04-26T19:59:47.293761+0000","last_became_active":"2024-04-26T19:59:45.290411+0000","last_became_peered":"2024-04-26T19:59:45.290411+0000","last_unstale":"2024-04-26T19:59:47.293761+0000","last_undegraded":"2024-04-26T19:59:47.293761+0000","last_fullsized":"2024-04-26T19:59:47.293761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:40.386765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464967+0000","last_change":"2024-04-26T19:59:46.288442+0000","last_active":"2024-04-26T19:59:47.464967+0000","last_peered":"2024-04-26T19:59:47.464967+0000","last_clean":"2024-04-26T19:59:47.464967+0000","last_became_active":"2024-04-26T19:59:46.288239+0000","last_became_peered":"2024-04-26T19:59:46.288239+0000","last_unstale":"2024-04-26T19:59:47.464967+0000","last_undegraded":"2024-04-26T19:59:47.464967+0000","last_fullsized":"2024-04-26T19:59:47.464967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:03.910392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.294034+0000","last_change":"2024-04-26T19:59:45.280986+0000","last_active":"2024-04-26T19:59:47.294034+0000","last_peered":"2024-04-26T19:59:47.294034+0000","last_clean":"2024-04-26T19:59:47.294034+0000","last_became_active":"2024-04-26T19:59:45.280606+0000","last_became_peered":"2024-04-26T19:59:45.280606+0000","last_unstale":"2024-04-26T19:59:47.294034+0000","last_undegraded":"2024-04-26T19:59:47.294034+0000","last_fullsized":"2024-04-26T19:59:47.294034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464007+0000","last_change":"2024-04-26T19:59:46.296268+0000","last_active":"2024-04-26T19:59:47.464007+0000","last_peered":"2024-04-26T19:59:47.464007+0000","last_clean":"2024-04-26T19:59:47.464007+0000","last_became_active":"2024-04-26T19:59:46.294912+0000","last_became_peered":"2024-04-26T19:59:46.294912+0000","last_unstale":"2024-04-26T19:59:47.464007+0000","last_undegraded":"2024-04-26T19:59:47.464007+0000","last_fullsized":"2024-04-26T19:59:47.464007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:57.904937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.469551+0000","last_change":"2024-04-26T19:59:45.299596+0000","last_active":"2024-04-26T19:59:47.469551+0000","last_peered":"2024-04-26T19:59:47.469551+0000","last_clean":"2024-04-26T19:59:47.469551+0000","last_became_active":"2024-04-26T19:59:45.299284+0000","last_became_peered":"2024-04-26T19:59:45.299284+0000","last_unstale":"2024-04-26T19:59:47.469551+0000","last_undegraded":"2024-04-26T19:59:47.469551+0000","last_fullsized":"2024-04-26T19:59:47.469551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:34.402472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.465783+0000","last_change":"2024-04-26T19:59:46.294465+0000","last_active":"2024-04-26T19:59:47.465783+0000","last_peered":"2024-04-26T19:59:47.465783+0000","last_clean":"2024-04-26T19:59:47.465783+0000","last_became_active":"2024-04-26T19:59:46.294138+0000","last_became_peered":"2024-04-26T19:59:46.294138+0000","last_unstale":"2024-04-26T19:59:47.465783+0000","last_undegraded":"2024-04-26T19:59:47.465783+0000","last_fullsized":"2024-04-26T19:59:47.465783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:31.037519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467698+0000","last_change":"2024-04-26T19:59:46.304096+0000","last_active":"2024-04-26T19:59:47.467698+0000","last_peered":"2024-04-26T19:59:47.467698+0000","last_clean":"2024-04-26T19:59:47.467698+0000","last_became_active":"2024-04-26T19:59:46.299682+0000","last_became_peered":"2024-04-26T19:59:46.299682+0000","last_unstale":"2024-04-26T19:59:47.467698+0000","last_undegraded":"2024-04-26T19:59:47.467698+0000","last_fullsized":"2024-04-26T19:59:47.467698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:59.302446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467128+0000","last_change":"2024-04-26T19:59:45.302197+0000","last_active":"2024-04-26T19:59:47.467128+0000","last_peered":"2024-04-26T19:59:47.467128+0000","last_clean":"2024-04-26T19:59:47.467128+0000","last_became_active":"2024-04-26T19:59:45.301818+0000","last_became_peered":"2024-04-26T19:59:45.301818+0000","last_unstale":"2024-04-26T19:59:47.467128+0000","last_undegraded":"2024-04-26T19:59:47.467128+0000","last_fullsized":"2024-04-26T19:59:47.467128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:43.563741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.466068+0000","last_change":"2024-04-26T19:59:46.309844+0000","last_active":"2024-04-26T19:59:47.466068+0000","last_peered":"2024-04-26T19:59:47.466068+0000","last_clean":"2024-04-26T19:59:47.466068+0000","last_became_active":"2024-04-26T19:59:46.309519+0000","last_became_peered":"2024-04-26T19:59:46.309519+0000","last_unstale":"2024-04-26T19:59:47.466068+0000","last_undegraded":"2024-04-26T19:59:47.466068+0000","last_fullsized":"2024-04-26T19:59:47.466068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:49.958273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.467758+0000","last_change":"2024-04-26T19:59:45.301972+0000","last_active":"2024-04-26T19:59:47.467758+0000","last_peered":"2024-04-26T19:59:47.467758+0000","last_clean":"2024-04-26T19:59:47.467758+0000","last_became_active":"2024-04-26T19:59:45.301689+0000","last_became_peered":"2024-04-26T19:59:45.301689+0000","last_unstale":"2024-04-26T19:59:47.467758+0000","last_undegraded":"2024-04-26T19:59:47.467758+0000","last_fullsized":"2024-04-26T19:59:47.467758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:44.042106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463870+0000","last_change":"2024-04-26T19:59:46.290774+0000","last_active":"2024-04-26T19:59:47.463870+0000","last_peered":"2024-04-26T19:59:47.463870+0000","last_clean":"2024-04-26T19:59:47.463870+0000","last_became_active":"2024-04-26T19:59:46.290172+0000","last_became_peered":"2024-04-26T19:59:46.290172+0000","last_unstale":"2024-04-26T19:59:47.463870+0000","last_undegraded":"2024-04-26T19:59:47.463870+0000","last_fullsized":"2024-04-26T19:59:47.463870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:37.919460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464753+0000","last_change":"2024-04-26T19:59:45.287597+0000","last_active":"2024-04-26T19:59:47.464753+0000","last_peered":"2024-04-26T19:59:47.464753+0000","last_clean":"2024-04-26T19:59:47.464753+0000","last_became_active":"2024-04-26T19:59:45.287273+0000","last_became_peered":"2024-04-26T19:59:45.287273+0000","last_unstale":"2024-04-26T19:59:47.464753+0000","last_undegraded":"2024-04-26T19:59:47.464753+0000","last_fullsized":"2024-04-26T19:59:47.464753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:13:58.169634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.463701+0000","last_change":"2024-04-26T19:59:46.301596+0000","last_active":"2024-04-26T19:59:47.463701+0000","last_peered":"2024-04-26T19:59:47.463701+0000","last_clean":"2024-04-26T19:59:47.463701+0000","last_became_active":"2024-04-26T19:59:46.292219+0000","last_became_peered":"2024-04-26T19:59:46.292219+0000","last_unstale":"2024-04-26T19:59:47.463701+0000","last_undegraded":"2024-04-26T19:59:47.463701+0000","last_fullsized":"2024-04-26T19:59:47.463701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:45.238672+0000","last_clean_scrub_stamp":"2024-04-26T19:59:45.238672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:15.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T19:59:47.464191+0000","last_change":"2024-04-26T19:59:45.293237+0000","last_active":"2024-04-26T19:59:47.464191+0000","last_peered":"2024-04-26T19:59:47.464191+0000","last_clean":"2024-04-26T19:59:47.464191+0000","last_became_active":"2024-04-26T19:59:45.292699+0000","last_became_peered":"2024-04-26T19:59:45.292699+0000","last_unstale":"2024-04-26T19:59:47.464191+0000","last_undegraded":"2024-04-26T19:59:47.464191+0000","last_fullsized":"2024-04-26T19:59:47.464191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:44.223039+0000","last_clean_scrub_stamp":"2024-04-26T19:59:44.223039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:26.361871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:59:57.606 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T19:59:57.606 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T19:59:57.606 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T19:59:57.606 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T19:59:57.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.839+0000 7f8ac1200640 1 Processor -- start 2024-04-26T19:59:57.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.839+0000 7f8ac1200640 1 -- start start 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8ac1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc072f30 0x7f8abc0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8ac1200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8abc0b1500 0x7f8abc0b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8ac1200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc0aff20 con 0x7f8abc15be20 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8ac1200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0afd00 con 0x7f8abc0b1500 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8ac1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0b0200 con 0x7f8abc072f30 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8abb400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8abc15be20 0x7f8abc0af5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52810/0 (socket says 172.21.15.88:52810) 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8abb400640 1 -- 172.21.15.88:0/2627813626 learned_addr learned my addr 172.21.15.88:0/2627813626 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:57.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.841+0000 7f8abbe00640 1 --2- 172.21.15.88:0/2627813626 >> v2:172.21.15.88:3300/0 conn(0x7f8abc0b1500 0x7f8abc0b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8ab2a00640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc072f30 0x7f8abc0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abaa00640 1 -- 172.21.15.88:0/2627813626 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 803361276 0 0) 0x7f8abc0aff20 con 0x7f8abc15be20 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abaa00640 1 -- 172.21.15.88:0/2627813626 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aa0003520 con 0x7f8abc15be20 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abbe00640 1 -- 172.21.15.88:0/2627813626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc072f30 msgr2=0x7f8abc0b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abbe00640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc072f30 0x7f8abc0b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abbe00640 1 -- 172.21.15.88:0/2627813626 >> v1:172.21.15.88:6789/0 conn(0x7f8abc15be20 legacy=0x7f8abc0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.842+0000 7f8abbe00640 1 -- 172.21.15.88:0/2627813626 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0b04e0 con 0x7f8abc0b1500 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.843+0000 7f8abbe00640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc0b1500 0x7f8abc0b58a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097a0 tx=0x7f8ab402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3993870b22e9705 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8abaa00640 1 -- 172.21.15.88:0/2627813626 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4037070 con 0x7f8abc0b1500 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8abaa00640 1 -- 172.21.15.88:0/2627813626 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4032440 con 0x7f8abc0b1500 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8abaa00640 1 -- 172.21.15.88:0/2627813626 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40327f0 con 0x7f8abc0b1500 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 -- 172.21.15.88:0/2627813626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc0b1500 msgr2=0x7f8abc0b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc0b1500 0x7f8abc0b58a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097a0 tx=0x7f8ab402a180 comp rx=0 tx=0).stop 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 -- 172.21.15.88:0/2627813626 shutdown_connections 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc0b1500 0x7f8abc0b58a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 --2- 172.21.15.88:0/2627813626 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc072f30 0x7f8abc0b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 -- 172.21.15.88:0/2627813626 >> 172.21.15.88:0/2627813626 conn(0x7f8abc06d460 msgr2=0x7f8abc06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 -- 172.21.15.88:0/2627813626 shutdown_connections 2024-04-26T19:59:57.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.844+0000 7f8ac1200640 1 -- 172.21.15.88:0/2627813626 wait complete. 2024-04-26T19:59:57.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.846+0000 7f8ac1200640 1 Processor -- start 2024-04-26T19:59:57.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.846+0000 7f8ac1200640 1 -- start start 2024-04-26T19:59:57.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 0x7f8abc0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 0x7f8abc15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b5de0 con 0x7f8abc072f30 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0b0aa0 con 0x7f8abc156960 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8ac1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc0b5f60 con 0x7f8abc15be20 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8abb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8abb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45310/0 (socket says 172.21.15.88:45310) 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.847+0000 7f8abb400640 1 -- 172.21.15.88:0/4265279031 learned_addr learned my addr 172.21.15.88:0/4265279031 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8ab2a00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 0x7f8abc0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abbe00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 0x7f8abc15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 -- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 msgr2=0x7f8abc0ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 0x7f8abc0ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 -- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 msgr2=0x7f8abc15dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 0x7f8abc15dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 -- 172.21.15.88:0/4265279031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc0d1920 con 0x7f8abc072f30 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8ab2a00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 0x7f8abc0ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:57.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abbe00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 0x7f8abc15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8abb400640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00caa0 tx=0x7f8aac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac017070 con 0x7f8abc072f30 2024-04-26T19:59:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac00ac00 con 0x7f8abc072f30 2024-04-26T19:59:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.848+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac012400 con 0x7f8abc072f30 2024-04-26T19:59:57.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.849+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc0d1c00 con 0x7f8abc072f30 2024-04-26T19:59:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.849+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc0d21c0 con 0x7f8abc072f30 2024-04-26T19:59:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.850+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc0d23c0 con 0x7f8abc072f30 2024-04-26T19:59:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.851+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8aac01a070 con 0x7f8abc072f30 2024-04-26T19:59:57.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.852+0000 7f8ab3e00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 0x7f8a9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:57.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.852+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8aac08b9a0 con 0x7f8abc072f30 2024-04-26T19:59:57.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.852+0000 7f8ab2a00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 0x7f8a9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:57.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.853+0000 7f8ab2a00640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 0x7f8a9806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0ce390 tx=0x7f8aa4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:57.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:57.857+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8abc0d23c0 con 0x7f8abc072f30 2024-04-26T19:59:58.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.088+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8abc0af6c0 con 0x7f8abc072f30 2024-04-26T19:59:58.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.089+0000 7f8ab3e00640 1 -- 172.21.15.88:0/4265279031 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8aac059d30 con 0x7f8abc072f30 2024-04-26T19:59:58.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:58.091 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T19:59:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 msgr2=0x7f8a9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 0x7f8a9806ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0ce390 tx=0x7f8aa4008040 comp rx=0 tx=0).stop 2024-04-26T19:59:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 msgr2=0x7f8abc0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00caa0 tx=0x7f8aac00cf60 comp rx=0 tx=0).stop 2024-04-26T19:59:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 shutdown_connections 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8abc156960 0x7f8abc15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a9806c920 0x7f8a9806ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8abc15be20 0x7f8abc0ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 --2- 172.21.15.88:0/4265279031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8abc072f30 0x7f8abc0d1340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.091+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 >> 172.21.15.88:0/4265279031 conn(0x7f8abc06d460 msgr2=0x7f8abc06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.092+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 shutdown_connections 2024-04-26T19:59:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.092+0000 7f8ac1200640 1 -- 172.21.15.88:0/4265279031 wait complete. 2024-04-26T19:59:58.110 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T19:59:58.110 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:59:58.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.294+0000 7f8951200640 1 Processor -- start 2024-04-26T19:59:58.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.294+0000 7f8951200640 1 -- start start 2024-04-26T19:59:58.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8951200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f894c06ce80 0x7f894c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8951200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f894c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8951200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c0af9a0 con 0x7f894c15cdf0 2024-04-26T19:59:58.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8951200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f894c0ae2c0 con 0x7f894c06ce80 2024-04-26T19:59:58.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8951200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f894c0b05b0 con 0x7f894c0cd650 2024-04-26T19:59:58.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8949e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f894c15cdf0 0x7f894c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52818/0 (socket says 172.21.15.88:52818) 2024-04-26T19:59:58.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8949e00640 1 -- 172.21.15.88:0/2944030717 learned_addr learned my addr 172.21.15.88:0/2944030717 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:58.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f894a800640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f894c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.296+0000 7f8949400640 1 --2- 172.21.15.88:0/2944030717 >> v2:172.21.15.132:3300/0 conn(0x7f894c06ce80 0x7f894c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644032432 0 0) 0x7f894c0af9a0 con 0x7f894c15cdf0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8930003520 con 0x7f894c15cdf0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 354537813 0 0) 0x7f8930003520 con 0x7f894c15cdf0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 msgr2=0x7f894c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f894c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 >> v2:172.21.15.132:3300/0 conn(0x7f894c06ce80 msgr2=0x7f894c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 --2- 172.21.15.88:0/2944030717 >> v2:172.21.15.132:3300/0 conn(0x7f894c06ce80 0x7f894c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c0afc80 con 0x7f894c15cdf0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f893c001670 con 0x7f894c15cdf0 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 >> v1:172.21.15.88:6789/0 conn(0x7f894c15cdf0 legacy=0x7f894c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89300039b0 0x7f8930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8930004800 0x7f8930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f89300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0b05b0 con 0x7f89300039b0 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f893000b9c0 con 0x7f8930004800 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f893000bb40 con 0x7f894c0cd650 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.297+0000 7f8949e00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89300039b0 0x7f8930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f894a800640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f89300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f8949e00640 1 -- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 msgr2=0x7f89300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f8949e00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c0cd650 0x7f89300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f8949e00640 1 -- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8930004800 msgr2=0x7f8930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f8949e00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8930004800 0x7f8930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.298+0000 7f8949e00640 1 -- 172.21.15.88:0/2944030717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930003520 con 0x7f89300039b0 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.299+0000 7f8949e00640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89300039b0 0x7f8930003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c240 tx=0x7f8930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24fa8fbae823ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.299+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c02e070 con 0x7f89300039b0 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.299+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893c005af0 con 0x7f89300039b0 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.299+0000 7f8948a00640 1 -- 172.21.15.88:0/2944030717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c007d30 con 0x7f89300039b0 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 -- 172.21.15.88:0/2944030717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89300039b0 msgr2=0x7f8930003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 --2- 172.21.15.88:0/2944030717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89300039b0 0x7f8930003dd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f893c00c240 tx=0x7f8930005830 comp rx=0 tx=0).stop 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f894a800640 1 -- 172.21.15.88:0/2944030717 reap_dead start 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 -- 172.21.15.88:0/2944030717 shutdown_connections 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 -- 172.21.15.88:0/2944030717 >> 172.21.15.88:0/2944030717 conn(0x7f894c0b2d50 msgr2=0x7f894c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 -- 172.21.15.88:0/2944030717 shutdown_connections 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.300+0000 7f8951200640 1 -- 172.21.15.88:0/2944030717 wait complete. 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.302+0000 7f8951200640 1 Processor -- start 2024-04-26T19:59:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.302+0000 7f8951200640 1 -- start start 2024-04-26T19:59:58.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89300039b0 0x7f894c09e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c152e90 0x7f894c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f894c15d1d0 con 0x7f894c0ce230 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0b0240 con 0x7f89300039b0 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8951200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f894c15d350 con 0x7f894c152e90 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8949400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8949400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45320/0 (socket says 172.21.15.88:45320) 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.303+0000 7f8949400640 1 -- 172.21.15.88:0/1623614360 learned_addr learned my addr 172.21.15.88:0/1623614360 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949e00640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89300039b0 0x7f894c09e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f894a800640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c152e90 0x7f894c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 -- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c152e90 msgr2=0x7f894c1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c152e90 0x7f894c1522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 -- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89300039b0 msgr2=0x7f894c09e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89300039b0 0x7f894c09e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 -- 172.21.15.88:0/1623614360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c09baf0 con 0x7f894c0ce230 2024-04-26T19:59:58.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8949400640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f893800e7a0 tx=0x7f893800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938017070 con 0x7f894c0ce230 2024-04-26T19:59:58.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893800c940 con 0x7f894c0ce230 2024-04-26T19:59:58.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.304+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800cd10 con 0x7f894c0ce230 2024-04-26T19:59:58.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.305+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c0ce6c0 con 0x7f894c0ce230 2024-04-26T19:59:58.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.305+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c0ce9a0 con 0x7f894c0ce230 2024-04-26T19:59:58.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.306+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c069540 con 0x7f894c0ce230 2024-04-26T19:59:58.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.307+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f893801a070 con 0x7f894c0ce230 2024-04-26T19:59:58.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.307+0000 7f8942000640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 0x7f892c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.308+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f893808b950 con 0x7f894c0ce230 2024-04-26T19:59:58.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.308+0000 7f8949e00640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 0x7f892c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.308+0000 7f8949e00640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 0x7f892c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f893c033000 tx=0x7f893c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.312+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8938059d80 con 0x7f894c0ce230 2024-04-26T19:59:58.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.499+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f894c158430 con 0x7f894c0ce230 2024-04-26T19:59:58.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.502+0000 7f8942000640 1 -- 172.21.15.88:0/1623614360 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f8938059680 con 0x7f894c0ce230 2024-04-26T19:59:58.504 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T19:59:58.504 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:59:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 msgr2=0x7f892c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 0x7f892c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f893c033000 tx=0x7f893c009030 comp rx=0 tx=0).stop 2024-04-26T19:59:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 msgr2=0x7f894c151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f893800e7a0 tx=0x7f893800ec60 comp rx=0 tx=0).stop 2024-04-26T19:59:58.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 shutdown_connections 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f892c06c8f0 0x7f892c06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f894c152e90 0x7f894c1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f894c0ce230 0x7f894c151d60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 --2- 172.21.15.88:0/1623614360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f89300039b0 0x7f894c09e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.504+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 >> 172.21.15.88:0/1623614360 conn(0x7f894c0b2d50 msgr2=0x7f894c15b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.505+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 shutdown_connections 2024-04-26T19:59:58.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.505+0000 7f8951200640 1 -- 172.21.15.88:0/1623614360 wait complete. 2024-04-26T19:59:58.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:59:58.524 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:59:58.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.721+0000 7f8d61800640 1 Processor -- start 2024-04-26T19:59:58.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.721+0000 7f8d61800640 1 -- start start 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d61800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d5c15cf40 0x7f8d5c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d61800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c15d970 con 0x7f8d5c15ee60 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d61800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c0b4fe0 con 0x7f8d5c15c450 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d61800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c0b52c0 con 0x7f8d5c15cf40 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d5b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8d5c15ee60 0x7f8d5c1639f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52824/0 (socket says 172.21.15.88:52824) 2024-04-26T19:59:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.723+0000 7f8d5b400640 1 -- 172.21.15.88:0/70991910 learned_addr learned my addr 172.21.15.88:0/70991910 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 493611929 0 0) 0x7f8d5c15d970 con 0x7f8d5c15ee60 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d60e00640 1 --2- 172.21.15.88:0/70991910 >> v2:172.21.15.132:3300/0 conn(0x7f8d5c15cf40 0x7f8d5c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d40005730 con 0x7f8d5c15ee60 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1230251352 0 0) 0x7f8d40005730 con 0x7f8d5c15ee60 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 >> v2:172.21.15.132:3300/0 conn(0x7f8d5c15cf40 msgr2=0x7f8d5c0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 --2- 172.21.15.88:0/70991910 >> v2:172.21.15.132:3300/0 conn(0x7f8d5c15cf40 0x7f8d5c0b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 >> v1:172.21.15.132:6789/0 conn(0x7f8d5c15c450 legacy=0x7f8d5c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0b55a0 con 0x7f8d5c15ee60 2024-04-26T19:59:58.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f8d48001670 con 0x7f8d5c15ee60 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 >> v1:172.21.15.88:6789/0 conn(0x7f8d5c15ee60 legacy=0x7f8d5c1639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 0x7f8d40006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d40006a10 0x7f8d40006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d400074f0 0x7f8d4000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b52c0 con 0x7f8d40005c30 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4000e440 con 0x7f8d400074f0 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.724+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4000e640 con 0x7f8d40006a10 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 0x7f8d40006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d60e00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d400074f0 0x7f8d4000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 -- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d40006a10 msgr2=0x7f8d40006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d40006a10 0x7f8d40006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 -- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d400074f0 msgr2=0x7f8d4000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d400074f0 0x7f8d4000dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d5be00640 1 -- 172.21.15.88:0/70991910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40005730 con 0x7f8d40005c30 2024-04-26T19:59:58.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.725+0000 7f8d60e00640 1 -- 172.21.15.88:0/70991910 reap_dead start 2024-04-26T19:59:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.726+0000 7f8d5be00640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 0x7f8d40006050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c001720 tx=0x7f8d40007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf7aba4466fed2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.726+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c001370 con 0x7f8d40005c30 2024-04-26T19:59:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.726+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0052d0 con 0x7f8d40005c30 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.726+0000 7f8d5aa00640 1 -- 172.21.15.88:0/70991910 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c02ab90 con 0x7f8d40005c30 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 -- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 msgr2=0x7f8d40006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 0x7f8d40006050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c001720 tx=0x7f8d40007a40 comp rx=0 tx=0).stop 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 -- 172.21.15.88:0/70991910 shutdown_connections 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 --2- 172.21.15.88:0/70991910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d40005c30 0x7f8d40006050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 -- 172.21.15.88:0/70991910 >> 172.21.15.88:0/70991910 conn(0x7f8d5c0b2900 msgr2=0x7f8d5c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 -- 172.21.15.88:0/70991910 shutdown_connections 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.727+0000 7f8d61800640 1 -- 172.21.15.88:0/70991910 wait complete. 2024-04-26T19:59:58.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.728+0000 7f8d61800640 1 Processor -- start 2024-04-26T19:59:58.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.729+0000 7f8d61800640 1 -- start start 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.729+0000 7f8d61800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d40005c30 0x7f8d5c145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.729+0000 7f8d61800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d5c0c64e0 0x7f8d5c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d61800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d61800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b4aa0 con 0x7f8d5c156090 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d61800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b4920 con 0x7f8d40005c30 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d61800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b4c20 con 0x7f8d5c0c64e0 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45328/0 (socket says 172.21.15.88:45328) 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 -- 172.21.15.88:0/1111494925 learned_addr learned my addr 172.21.15.88:0/1111494925 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d5b400640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d5c0c64e0 0x7f8d5c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d5be00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d40005c30 0x7f8d5c145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 -- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d5c0c64e0 msgr2=0x7f8d5c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d5c0c64e0 0x7f8d5c0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 -- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d40005c30 msgr2=0x7f8d5c145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d40005c30 0x7f8d5c145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 -- 172.21.15.88:0/1111494925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c145bc0 con 0x7f8d5c156090 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.730+0000 7f8d60e00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009840 tx=0x7f8d54009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.731+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54019070 con 0x7f8d5c156090 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.731+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d54007d20 con 0x7f8d5c156090 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.731+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54014640 con 0x7f8d5c156090 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.731+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c157100 con 0x7f8d5c156090 2024-04-26T19:59:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.731+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c1576c0 con 0x7f8d5c156090 2024-04-26T19:59:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.732+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c0693c0 con 0x7f8d5c156090 2024-04-26T19:59:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.733+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d5401c070 con 0x7f8d5c156090 2024-04-26T19:59:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.734+0000 7f8d53e00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 0x7f8d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.734+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d5408b6f0 con 0x7f8d5c156090 2024-04-26T19:59:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.734+0000 7f8d5be00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 0x7f8d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:58.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.735+0000 7f8d5be00640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 0x7f8d3c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c032000 tx=0x7f8d4c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:58.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.738+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d5c145bc0 con 0x7f8d5c156090 2024-04-26T19:59:58.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.944+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d5c070000 con 0x7f8d5c156090 2024-04-26T19:59:58.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.945+0000 7f8d53e00640 1 -- 172.21.15.88:0/1111494925 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8d54059a80 con 0x7f8d5c156090 2024-04-26T19:59:58.947 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.948+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 msgr2=0x7f8d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.948+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 0x7f8d3c06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c032000 tx=0x7f8d4c02e000 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.948+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 msgr2=0x7f8d5c156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.948+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009840 tx=0x7f8d54009d00 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 shutdown_connections 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d5c156090 0x7f8d5c156510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3c06c8f0 0x7f8d3c06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d5c0c64e0 0x7f8d5c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 --2- 172.21.15.88:0/1111494925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d40005c30 0x7f8d5c145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 >> 172.21.15.88:0/1111494925 conn(0x7f8d5c0b2900 msgr2=0x7f8d5c15ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 shutdown_connections 2024-04-26T19:59:58.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:58.949+0000 7f8d61800640 1 -- 172.21.15.88:0/1111494925 wait complete. 2024-04-26T19:59:58.968 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:59:59.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.159+0000 7fd01b000640 1 Processor -- start 2024-04-26T19:59:59.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.159+0000 7fd01b000640 1 -- start start 2024-04-26T19:59:59.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fd0140b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01b000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140b4d20 con 0x7fd014146280 2024-04-26T19:59:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01b000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140b5000 con 0x7fd014161140 2024-04-26T19:59:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd01406e5f0 con 0x7fd014160d60 2024-04-26T19:59:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01a600640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd014146280 0x7fd0140cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49820/0 (socket says 172.21.15.88:49820) 2024-04-26T19:59:59.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.161+0000 7fd01a600640 1 -- 172.21.15.88:0/2052387441 learned_addr learned my addr 172.21.15.88:0/2052387441 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T19:59:59.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd019c00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fd0140b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939464119 0 0) 0x7fd0140b4d20 con 0x7fd014146280 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcffc003520 con 0x7fd014146280 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4064218987 0 0) 0x7fcffc003520 con 0x7fd014146280 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 msgr2=0x7fd0140b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fd0140b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 >> v1:172.21.15.132:6789/0 conn(0x7fd014161140 legacy=0x7fd01406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140b52e0 con 0x7fd014146280 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd00c0019f0 con 0x7fd014146280 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 >> v1:172.21.15.88:6789/0 conn(0x7fd014146280 legacy=0x7fd0140cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fcffc003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fcffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcffc004db0 0x7fcffc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd01406e5f0 con 0x7fcffc0040d0 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc00bc20 con 0x7fcffc004db0 2024-04-26T19:59:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc00be20 con 0x7fd014160d60 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.162+0000 7fd019200640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fcffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019c00640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fcffc003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd01a600640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcffc004db0 0x7fcffc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019200640 1 -- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 msgr2=0x7fcffc003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019200640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd014160d60 0x7fcffc003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019200640 1 -- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcffc004db0 msgr2=0x7fcffc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019200640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcffc004db0 0x7fcffc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd019200640 1 -- 172.21.15.88:0/2052387441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc003520 con 0x7fcffc0040d0 2024-04-26T19:59:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.163+0000 7fd01a600640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcffc004db0 0x7fcffc005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.164+0000 7fd019200640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fcffc004800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd000001750 tx=0x7fcffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2935d7107a5880 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.164+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000001370 con 0x7fcffc0040d0 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.164+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0000052d0 con 0x7fcffc0040d0 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.164+0000 7fd013e00640 1 -- 172.21.15.88:0/2052387441 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00002abe0 con 0x7fcffc0040d0 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 -- 172.21.15.88:0/2052387441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 msgr2=0x7fcffc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 --2- 172.21.15.88:0/2052387441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fcffc004800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd000001750 tx=0x7fcffc005830 comp rx=0 tx=0).stop 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01a600640 1 -- 172.21.15.88:0/2052387441 reap_dead start 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 -- 172.21.15.88:0/2052387441 shutdown_connections 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 -- 172.21.15.88:0/2052387441 >> 172.21.15.88:0/2052387441 conn(0x7fd0140b2900 msgr2=0x7fd0140b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 -- 172.21.15.88:0/2052387441 shutdown_connections 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.165+0000 7fd01b000640 1 -- 172.21.15.88:0/2052387441 wait complete. 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.167+0000 7fd01b000640 1 Processor -- start 2024-04-26T19:59:59.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.167+0000 7fd01b000640 1 -- start start 2024-04-26T19:59:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd01415d560 0x7fd0140cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0140cb510 0x7fd0140cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd014161770 con 0x7fcffc0040d0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd0140cf6f0 con 0x7fd01415d560 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01b000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd0141618f0 con 0x7fd0140cb510 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33938/0 (socket says 172.21.15.88:33938) 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 -- 172.21.15.88:0/2249210463 learned_addr learned my addr 172.21.15.88:0/2249210463 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd01a600640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0140cb510 0x7fd0140cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 -- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0140cb510 msgr2=0x7fd0140cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0140cb510 0x7fd0140cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 -- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd01415d560 msgr2=0x7fd0140cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019200640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd01415d560 0x7fd0140cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd01415d560 0x7fd0140cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.168+0000 7fd019c00640 1 -- 172.21.15.88:0/2249210463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140aa700 con 0x7fcffc0040d0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd019c00640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd00800f5e0 tx=0x7fd00800faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008019020 con 0x7fcffc0040d0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd008014870 con 0x7fcffc0040d0 2024-04-26T19:59:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008014c60 con 0x7fcffc0040d0 2024-04-26T19:59:59.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0140aa9e0 con 0x7fcffc0040d0 2024-04-26T19:59:59.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.169+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01414f9a0 con 0x7fcffc0040d0 2024-04-26T19:59:59.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.170+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01414f4b0 con 0x7fcffc0040d0 2024-04-26T19:59:59.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.171+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd00801c020 con 0x7fcffc0040d0 2024-04-26T19:59:59.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.172+0000 7fd011600640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.172+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd00808c350 con 0x7fcffc0040d0 2024-04-26T19:59:59.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.172+0000 7fd019200640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.173+0000 7fd019200640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 0x7fcff406edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd000032000 tx=0x7fd00002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.176+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd008023020 con 0x7fcffc0040d0 2024-04-26T19:59:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.360+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd0140693c0 con 0x7fcffc0040d0 2024-04-26T19:59:59.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.362+0000 7fd011600640 1 -- 172.21.15.88:0/2249210463 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd008026350 con 0x7fcffc0040d0 2024-04-26T19:59:59.363 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 msgr2=0x7fcff406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 0x7fcff406edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd000032000 tx=0x7fd00002e000 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 msgr2=0x7fd0140cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd00800f5e0 tx=0x7fd00800faa0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 shutdown_connections 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcff406c8f0 0x7fcff406edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd0140cb510 0x7fd0140cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd01415d560 0x7fd0140cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 --2- 172.21.15.88:0/2249210463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcffc0040d0 0x7fd0140cc8d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 >> 172.21.15.88:0/2249210463 conn(0x7fd0140b2900 msgr2=0x7fd014072120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:59.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 shutdown_connections 2024-04-26T19:59:59.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.364+0000 7fd01b000640 1 -- 172.21.15.88:0/2249210463 wait complete. 2024-04-26T19:59:59.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:59:59.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.580+0000 7fb5aa600640 1 Processor -- start 2024-04-26T19:59:59.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.580+0000 7fb5aa600640 1 -- start start 2024-04-26T19:59:59.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5aa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a4156650 0x7fb5a415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5aa600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5a406c770 0x7fb5a406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5aa600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5a406eea0 con 0x7fb5a4073120 2024-04-26T19:59:59.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5aa600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb5a40b4c40 con 0x7fb5a406c770 2024-04-26T19:59:59.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5aa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a406f140 con 0x7fb5a4156650 2024-04-26T19:59:59.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5a2a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5a406c770 0x7fb5a406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5a2a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5a406c770 0x7fb5a406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33952/0 (socket says 172.21.15.88:33952) 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5a2a00640 1 -- 172.21.15.88:0/2000765322 learned_addr learned my addr 172.21.15.88:0/2000765322 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.582+0000 7fb5a3400640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a4156650 0x7fb5a415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.583+0000 7fb5a2a00640 1 -- 172.21.15.88:0/2000765322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a4156650 msgr2=0x7fb5a415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.583+0000 7fb5a2a00640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a4156650 0x7fb5a415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.583+0000 7fb5a2a00640 1 -- 172.21.15.88:0/2000765322 >> v1:172.21.15.132:6789/0 conn(0x7fb5a4073120 legacy=0x7fb5a40735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.583+0000 7fb5a2a00640 1 -- 172.21.15.88:0/2000765322 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a406f420 con 0x7fb5a406c770 2024-04-26T19:59:59.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.584+0000 7fb5a2a00640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a406c770 0x7fb5a406cbf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb590009870 tx=0x7fb59002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b5a1915831a4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.584+0000 7fb5a2000640 1 -- 172.21.15.88:0/2000765322 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590037070 con 0x7fb5a406c770 2024-04-26T19:59:59.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.584+0000 7fb5a2000640 1 -- 172.21.15.88:0/2000765322 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb590032440 con 0x7fb5a406c770 2024-04-26T19:59:59.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.584+0000 7fb5a2000640 1 -- 172.21.15.88:0/2000765322 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590032810 con 0x7fb5a406c770 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 -- 172.21.15.88:0/2000765322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a406c770 msgr2=0x7fb5a406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a406c770 0x7fb5a406cbf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb590009870 tx=0x7fb59002a270 comp rx=0 tx=0).stop 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 -- 172.21.15.88:0/2000765322 shutdown_connections 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a406c770 0x7fb5a406cbf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 --2- 172.21.15.88:0/2000765322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a4156650 0x7fb5a415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 -- 172.21.15.88:0/2000765322 >> 172.21.15.88:0/2000765322 conn(0x7fb5a40b2d50 msgr2=0x7fb5a40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 -- 172.21.15.88:0/2000765322 shutdown_connections 2024-04-26T19:59:59.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.585+0000 7fb5aa600640 1 -- 172.21.15.88:0/2000765322 wait complete. 2024-04-26T19:59:59.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.586+0000 7fb5aa600640 1 Processor -- start 2024-04-26T19:59:59.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 -- start start 2024-04-26T19:59:59.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 0x7fb5a41395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5a4073120 0x7fb5a4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a406fcc0 con 0x7fb5a4156650 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a406f9e0 con 0x7fb5a4073120 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.587+0000 7fb5aa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a415d320 con 0x7fb5a406c770 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33960/0 (socket says 172.21.15.88:33960) 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 -- 172.21.15.88:0/1024826769 learned_addr learned my addr 172.21.15.88:0/1024826769 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a2a00640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5a4073120 0x7fb5a4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3400640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 0x7fb5a41395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 -- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 msgr2=0x7fb5a41395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 0x7fb5a41395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 -- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5a4073120 msgr2=0x7fb5a4156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5a4073120 0x7fb5a4156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3e00640 1 -- 172.21.15.88:0/1024826769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a4159240 con 0x7fb5a4156650 2024-04-26T19:59:59.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.588+0000 7fb5a3400640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 0x7fb5a41395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb5a3e00640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c7a0 tx=0x7fb59c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c003b70 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c003d30 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00aa90 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a4159520 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.589+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a40af0e0 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.590+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a40af5f0 con 0x7fb5a4156650 2024-04-26T19:59:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.591+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb59c00ac30 con 0x7fb5a4156650 2024-04-26T19:59:59.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.592+0000 7fb59b400640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:59:59.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.592+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb59c08bb50 con 0x7fb5a4156650 2024-04-26T19:59:59.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.592+0000 7fb5a3400640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:59:59.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.593+0000 7fb5a3400640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 0x7fb58406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb594006fd0 tx=0x7fb594008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:59:59.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.597+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5a40af5f0 con 0x7fb5a4156650 2024-04-26T19:59:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.794+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5a4069650 con 0x7fb5a4156650 2024-04-26T19:59:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.795+0000 7fb59b400640 1 -- 172.21.15.88:0/1024826769 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fb59c059ee0 con 0x7fb5a4156650 2024-04-26T19:59:59.796 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 msgr2=0x7fb58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 0x7fb58406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb594006fd0 tx=0x7fb594008040 comp rx=0 tx=0).stop 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 msgr2=0x7fb5a40af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c7a0 tx=0x7fb59c00cc60 comp rx=0 tx=0).stop 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 shutdown_connections 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4156650 0x7fb5a40af910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5a4073120 0x7fb5a4156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb58406c8f0 0x7fb58406edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 --2- 172.21.15.88:0/1024826769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5a406c770 0x7fb5a41395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:59:59.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 >> 172.21.15.88:0/1024826769 conn(0x7fb5a40b2d50 msgr2=0x7fb5a40729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:59:59.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.797+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 shutdown_connections 2024-04-26T19:59:59.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T19:59:59.798+0000 7fb5aa600640 1 -- 172.21.15.88:0/1024826769 wait complete. 2024-04-26T19:59:59.817 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:00.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.017+0000 7f352ba00640 1 Processor -- start 2024-04-26T20:00:00.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.018+0000 7f352ba00640 1 -- start start 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f352ba00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f352406c770 0x7f352406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f352ba00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f352415cfd0 0x7f3524073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f352ba00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352406d090 con 0x7f352415cb70 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f352ba00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f35240af460 con 0x7f352406c770 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f352ba00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f35240ae850 con 0x7f352415cfd0 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f352406c770 0x7f352406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f352406c770 0x7f352406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33968/0 (socket says 172.21.15.88:33968) 2024-04-26T20:00:00.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f351be00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f352415cb70 0x7f3524073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49826/0 (socket says 172.21.15.88:49826) 2024-04-26T20:00:00.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 -- 172.21.15.88:0/2601815908 learned_addr learned my addr 172.21.15.88:0/2601815908 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529a00640 1 --2- 172.21.15.88:0/2601815908 >> v2:172.21.15.132:3300/0 conn(0x7f352415cfd0 0x7f3524073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 -- 172.21.15.88:0/2601815908 >> v2:172.21.15.132:3300/0 conn(0x7f352415cfd0 msgr2=0x7f3524073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 --2- 172.21.15.88:0/2601815908 >> v2:172.21.15.132:3300/0 conn(0x7f352415cfd0 0x7f3524073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 -- 172.21.15.88:0/2601815908 >> v1:172.21.15.88:6789/0 conn(0x7f352415cb70 legacy=0x7f3524073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3523e00640 1 -- 172.21.15.88:0/2601815908 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106655021 0 0) 0x7f352406d090 con 0x7f352415cb70 2024-04-26T20:00:00.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.020+0000 7f3529000640 1 -- 172.21.15.88:0/2601815908 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35240aeb30 con 0x7f352406c770 2024-04-26T20:00:00.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.021+0000 7f3529000640 1 --2- 172.21.15.88:0/2601815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352406c770 0x7f352406cb50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3524074490 tx=0x7f351402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2b296326a9445a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.021+0000 7f3523e00640 1 -- 172.21.15.88:0/2601815908 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514037070 con 0x7f352406c770 2024-04-26T20:00:00.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.021+0000 7f3523e00640 1 -- 172.21.15.88:0/2601815908 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3514032440 con 0x7f352406c770 2024-04-26T20:00:00.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.021+0000 7f3523e00640 1 -- 172.21.15.88:0/2601815908 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514032810 con 0x7f352406c770 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 -- 172.21.15.88:0/2601815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352406c770 msgr2=0x7f352406cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 --2- 172.21.15.88:0/2601815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352406c770 0x7f352406cb50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3524074490 tx=0x7f351402a270 comp rx=0 tx=0).stop 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 -- 172.21.15.88:0/2601815908 shutdown_connections 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 --2- 172.21.15.88:0/2601815908 >> v2:172.21.15.132:3300/0 conn(0x7f352415cfd0 0x7f3524073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 --2- 172.21.15.88:0/2601815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352406c770 0x7f352406cb50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 -- 172.21.15.88:0/2601815908 >> 172.21.15.88:0/2601815908 conn(0x7f35240b2d50 msgr2=0x7f35240b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 -- 172.21.15.88:0/2601815908 shutdown_connections 2024-04-26T20:00:00.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.022+0000 7f352ba00640 1 -- 172.21.15.88:0/2601815908 wait complete. 2024-04-26T20:00:00.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.024+0000 7f352ba00640 1 Processor -- start 2024-04-26T20:00:00.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.024+0000 7f352ba00640 1 -- start start 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 0x7f3524150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f352415cfd0 0x7f3524159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f352415c600 con 0x7f352415cb70 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f35240af0f0 con 0x7f352406c770 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f352ba00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f352415c780 con 0x7f352415cfd0 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33980/0 (socket says 172.21.15.88:33980) 2024-04-26T20:00:00.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 -- 172.21.15.88:0/4081184679 learned_addr learned my addr 172.21.15.88:0/4081184679 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f3529000640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 0x7f3524150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f3529a00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f352415cfd0 0x7f3524159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 -- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f352415cfd0 msgr2=0x7f3524159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f352415cfd0 0x7f3524159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 -- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 msgr2=0x7f3524150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 0x7f3524150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.025+0000 7f351be00640 1 -- 172.21.15.88:0/4081184679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352409e4c0 con 0x7f352415cb70 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f3529000640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 0x7f3524150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f351be00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c500 tx=0x7f350c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c018070 con 0x7f352415cb70 2024-04-26T20:00:00.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f350c007580 con 0x7f352415cb70 2024-04-26T20:00:00.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c004a20 con 0x7f352415cb70 2024-04-26T20:00:00.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352409e760 con 0x7f352415cb70 2024-04-26T20:00:00.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.026+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352409ed20 con 0x7f352415cb70 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.028+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524069540 con 0x7f352415cb70 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.028+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f350c01b070 con 0x7f352415cb70 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.029+0000 7f3521600640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 0x7f350006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.029+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f350c08adc0 con 0x7f352415cb70 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.029+0000 7f3529000640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 0x7f350006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.030+0000 7f3529000640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 0x7f350006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3514034000 tx=0x7f351402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.034+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f350c059280 con 0x7f352415cb70 2024-04-26T20:00:00.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.271+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35240d0680 con 0x7f352415cb70 2024-04-26T20:00:00.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.273+0000 7f3521600640 1 -- 172.21.15.88:0/4081184679 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f350c058b80 con 0x7f352415cb70 2024-04-26T20:00:00.275 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:00.275 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:00.275 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:00.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 msgr2=0x7f350006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 0x7f350006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3514034000 tx=0x7f351402e000 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 msgr2=0x7f352406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c500 tx=0x7f350c00c9c0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 shutdown_connections 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f350006c8f0 0x7f350006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f352415cfd0 0x7f3524159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f352415cb70 0x7f352406f0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 --2- 172.21.15.88:0/4081184679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f352406c770 0x7f3524150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 >> 172.21.15.88:0/4081184679 conn(0x7f35240b2d50 msgr2=0x7f35240b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 shutdown_connections 2024-04-26T20:00:00.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.276+0000 7f352ba00640 1 -- 172.21.15.88:0/4081184679 wait complete. 2024-04-26T20:00:00.297 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:00.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 Processor -- start 2024-04-26T20:00:00.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 -- start start 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad98146680 0x7fad980b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fad980d0e70 0x7fad980d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fad98160930 0x7fad98160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fad9806e970 con 0x7fad980d0e70 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fad980d17f0 con 0x7fad98160930 2024-04-26T20:00:00.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.494+0000 7fad9ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fad98146a60 con 0x7fad98146680 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fad980d0e70 0x7fad980d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fad980d0e70 0x7fad980d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33986/0 (socket says 172.21.15.88:33986) 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 -- 172.21.15.88:0/1079094422 learned_addr learned my addr 172.21.15.88:0/1079094422 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad97e00640 1 --2- 172.21.15.88:0/1079094422 >> v2:172.21.15.132:3300/0 conn(0x7fad98160930 0x7fad98160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad97400640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad98146680 0x7fad980b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 -- 172.21.15.88:0/1079094422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad98146680 msgr2=0x7fad980b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad98146680 0x7fad980b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 -- 172.21.15.88:0/1079094422 >> v2:172.21.15.132:3300/0 conn(0x7fad98160930 msgr2=0x7fad98160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 --2- 172.21.15.88:0/1079094422 >> v2:172.21.15.132:3300/0 conn(0x7fad98160930 0x7fad98160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.495+0000 7fad96a00640 1 -- 172.21.15.88:0/1079094422 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9806dd60 con 0x7fad980d0e70 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad96a00640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad980d0e70 0x7fad980d12b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fad980d1ad0 tx=0x7fad8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd94dd1f3a4ada0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad96000640 1 -- 172.21.15.88:0/1079094422 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84037070 con 0x7fad980d0e70 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad96000640 1 -- 172.21.15.88:0/1079094422 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84032440 con 0x7fad980d0e70 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad96000640 1 -- 172.21.15.88:0/1079094422 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84032810 con 0x7fad980d0e70 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad9ce00640 1 -- 172.21.15.88:0/1079094422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad980d0e70 msgr2=0x7fad980d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.497+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad980d0e70 0x7fad980d12b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fad980d1ad0 tx=0x7fad8402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 -- 172.21.15.88:0/1079094422 shutdown_connections 2024-04-26T20:00:00.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1079094422 >> v2:172.21.15.132:3300/0 conn(0x7fad98160930 0x7fad98160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad980d0e70 0x7fad980d12b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1079094422 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad98146680 0x7fad980b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 -- 172.21.15.88:0/1079094422 >> 172.21.15.88:0/1079094422 conn(0x7fad9806c740 msgr2=0x7fad9806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 -- 172.21.15.88:0/1079094422 shutdown_connections 2024-04-26T20:00:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.498+0000 7fad9ce00640 1 -- 172.21.15.88:0/1079094422 wait complete. 2024-04-26T20:00:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.500+0000 7fad9ce00640 1 Processor -- start 2024-04-26T20:00:00.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.500+0000 7fad9ce00640 1 -- start start 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad980d0e70 0x7fad981f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad98146680 0x7fad981f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad9806e600 con 0x7fad98160930 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fad9806e320 con 0x7fad98146680 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad9ce00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fad980cd6a0 con 0x7fad980d0e70 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33992/0 (socket says 172.21.15.88:33992) 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 -- 172.21.15.88:0/1890947868 learned_addr learned my addr 172.21.15.88:0/1890947868 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad96a00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad98146680 0x7fad981f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97400640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad980d0e70 0x7fad981f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 -- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad980d0e70 msgr2=0x7fad981f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad980d0e70 0x7fad981f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 -- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad98146680 msgr2=0x7fad981f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad98146680 0x7fad981f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.501+0000 7fad97e00640 1 -- 172.21.15.88:0/1890947868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad98142fe0 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad97e00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fad90007e60 tx=0x7fad9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90014420 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90019070 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90014820 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad981432c0 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.502+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad98143880 con 0x7fad98160930 2024-04-26T20:00:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.503+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad980693c0 con 0x7fad98160930 2024-04-26T20:00:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.504+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad900149c0 con 0x7fad98160930 2024-04-26T20:00:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.505+0000 7fad8f400640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 0x7fad7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.505+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad9008c3b0 con 0x7fad98160930 2024-04-26T20:00:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.505+0000 7fad97400640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 0x7fad7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.506+0000 7fad97400640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 0x7fad7806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.509+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad9005a7e0 con 0x7fad98160930 2024-04-26T20:00:00.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.710+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad9815ebd0 con 0x7fad98160930 2024-04-26T20:00:00.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.713+0000 7fad8f400640 1 -- 172.21.15.88:0/1890947868 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7fad9005a0e0 con 0x7fad98160930 2024-04-26T20:00:00.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:00.714 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 msgr2=0x7fad7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 0x7fad7806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 msgr2=0x7fad98072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fad90007e60 tx=0x7fad9000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 shutdown_connections 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad98160930 0x7fad98072e90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad98146680 0x7fad981f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad7806c8f0 0x7fad7806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 --2- 172.21.15.88:0/1890947868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad980d0e70 0x7fad981f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 >> 172.21.15.88:0/1890947868 conn(0x7fad9806c740 msgr2=0x7fad980d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:00.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 shutdown_connections 2024-04-26T20:00:00.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.716+0000 7fad9ce00640 1 -- 172.21.15.88:0/1890947868 wait complete. 2024-04-26T20:00:00.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:00.738 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:00.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.939+0000 7f78a1c00640 1 Processor -- start 2024-04-26T20:00:00.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.939+0000 7f78a1c00640 1 -- start start 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f78a1c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f789c0d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f78a1c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f789c0d1da0 0x7f789c162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f78a1c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789c0d3830 con 0x7f789c0b6170 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f78a1c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f789c0d40d0 con 0x7f789c0d1da0 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f78a1c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f789c0d3b10 con 0x7f789c0d19c0 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f789a000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f789c0b6170 0x7f789c0c8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49842/0 (socket says 172.21.15.88:49842) 2024-04-26T20:00:00.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.940+0000 7f789a000640 1 -- 172.21.15.88:0/521459154 learned_addr learned my addr 172.21.15.88:0/521459154 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f789b400640 1 --2- 172.21.15.88:0/521459154 >> v2:172.21.15.132:3300/0 conn(0x7f789c0d1da0 0x7f789c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f789aa00640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f789c0d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1922735761 0 0) 0x7f789c0d3830 con 0x7f789c0b6170 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7880003520 con 0x7f789c0b6170 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 279825655 0 0) 0x7f7880003520 con 0x7f789c0b6170 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 msgr2=0x7f789c0d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f789c0d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 >> v2:172.21.15.132:3300/0 conn(0x7f789c0d1da0 msgr2=0x7f789c162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 --2- 172.21.15.88:0/521459154 >> v2:172.21.15.132:3300/0 conn(0x7f789c0d1da0 0x7f789c162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c0d3df0 con 0x7f789c0b6170 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f78840019f0 con 0x7f789c0b6170 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 >> v1:172.21.15.88:6789/0 conn(0x7f789c0b6170 legacy=0x7f789c0c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.941+0000 7f7899600640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78800039b0 0x7f7880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f7899600640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f7880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f7899600640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f78800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f789c0d3b10 con 0x7f7880004800 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f788000b9c0 con 0x7f78800039b0 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f788000bb40 con 0x7f789c0d19c0 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f7880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789b400640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f78800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 -- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 msgr2=0x7f78800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c0d19c0 0x7f78800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 -- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78800039b0 msgr2=0x7f7880003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78800039b0 0x7f7880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.942+0000 7f789a000640 1 -- 172.21.15.88:0/521459154 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7880003520 con 0x7f7880004800 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.943+0000 7f789a000640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f7880004c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f788400a8d0 tx=0x7f7880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f569331dc594ee8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788400a4c0 con 0x7f7880004800 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788402f070 con 0x7f7880004800 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f7899600640 1 -- 172.21.15.88:0/521459154 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788402bd00 con 0x7f7880004800 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 -- 172.21.15.88:0/521459154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 msgr2=0x7f7880004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 --2- 172.21.15.88:0/521459154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f7880004c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f788400a8d0 tx=0x7f7880005830 comp rx=0 tx=0).stop 2024-04-26T20:00:00.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f789b400640 1 -- 172.21.15.88:0/521459154 reap_dead start 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 -- 172.21.15.88:0/521459154 shutdown_connections 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 -- 172.21.15.88:0/521459154 >> 172.21.15.88:0/521459154 conn(0x7f789c09e460 msgr2=0x7f789c0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 -- 172.21.15.88:0/521459154 shutdown_connections 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.944+0000 7f78a1c00640 1 -- 172.21.15.88:0/521459154 wait complete. 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.946+0000 7f78a1c00640 1 Processor -- start 2024-04-26T20:00:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.946+0000 7f78a1c00640 1 -- start start 2024-04-26T20:00:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c157370 0x7f789c0b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c1577f0 0x7f789c0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f789c162d00 con 0x7f7880004800 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f789c162b80 con 0x7f789c1577f0 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f78a1c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f789c162e80 con 0x7f789c157370 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f789aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f789aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34000/0 (socket says 172.21.15.88:34000) 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.947+0000 7f789aa00640 1 -- 172.21.15.88:0/2725739903 learned_addr learned my addr 172.21.15.88:0/2725739903 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789a000640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c157370 0x7f789c0b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 -- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c157370 msgr2=0x7f789c0b8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c157370 0x7f789c0b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 -- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c1577f0 msgr2=0x7f789c0b9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c1577f0 0x7f789c0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 -- 172.21.15.88:0/2725739903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c155be0 con 0x7f7880004800 2024-04-26T20:00:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f789aa00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e7a0 tx=0x7f788c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c017070 con 0x7f7880004800 2024-04-26T20:00:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788c00c8c0 con 0x7f7880004800 2024-04-26T20:00:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00cc90 con 0x7f7880004800 2024-04-26T20:00:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.948+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c157d50 con 0x7f7880004800 2024-04-26T20:00:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.949+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c158310 con 0x7f7880004800 2024-04-26T20:00:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.950+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789c069650 con 0x7f7880004800 2024-04-26T20:00:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.951+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f788c01a070 con 0x7f7880004800 2024-04-26T20:00:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.952+0000 7f7892a00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 0x7f787806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.952+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f788c08b900 con 0x7f7880004800 2024-04-26T20:00:00.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.952+0000 7f789a000640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 0x7f787806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:00.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.953+0000 7f789a000640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 0x7f787806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7884033000 tx=0x7f7884007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:00.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:00.957+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f788c059d20 con 0x7f7880004800 2024-04-26T20:00:01.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.155+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f789c0d26d0 con 0x7f7880004800 2024-04-26T20:00:01.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.156+0000 7f7892a00640 1 -- 172.21.15.88:0/2725739903 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f788c08b020 con 0x7f7880004800 2024-04-26T20:00:01.157 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 msgr2=0x7f787806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 0x7f787806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7884033000 tx=0x7f7884007000 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 msgr2=0x7f789c1556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e7a0 tx=0x7f788c00ec60 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 shutdown_connections 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c1577f0 0x7f789c0b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f787806c8f0 0x7f787806edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f789c157370 0x7f789c0b8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 --2- 172.21.15.88:0/2725739903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7880004800 0x7f789c1556a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 >> 172.21.15.88:0/2725739903 conn(0x7f789c09e460 msgr2=0x7f789c06cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 shutdown_connections 2024-04-26T20:00:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.159+0000 7f78a1c00640 1 -- 172.21.15.88:0/2725739903 wait complete. 2024-04-26T20:00:01.180 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:01.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.379+0000 7fb477200640 1 Processor -- start 2024-04-26T20:00:01.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.379+0000 7fb477200640 1 -- start start 2024-04-26T20:00:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb477200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb47815ee60 0x7fb4780b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb477200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb47815cf40 0x7fb47806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb477200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb47815d970 con 0x7fb47815c450 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb477200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb47806e520 con 0x7fb47815cf40 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb477200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb4780b4fe0 con 0x7fb47815ee60 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb476800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb47815cf40 0x7fb47806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb476800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb47815cf40 0x7fb47806dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34008/0 (socket says 172.21.15.88:34008) 2024-04-26T20:00:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb476800640 1 -- 172.21.15.88:0/2495626359 learned_addr learned my addr 172.21.15.88:0/2495626359 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb475400640 1 --2- 172.21.15.88:0/2495626359 >> v2:172.21.15.132:3300/0 conn(0x7fb47815ee60 0x7fb4780b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb476800640 1 -- 172.21.15.88:0/2495626359 >> v2:172.21.15.132:3300/0 conn(0x7fb47815ee60 msgr2=0x7fb4780b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.381+0000 7fb476800640 1 --2- 172.21.15.88:0/2495626359 >> v2:172.21.15.132:3300/0 conn(0x7fb47815ee60 0x7fb4780b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.382+0000 7fb476800640 1 -- 172.21.15.88:0/2495626359 >> v1:172.21.15.132:6789/0 conn(0x7fb47815c450 legacy=0x7fb47815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.382+0000 7fb476800640 1 -- 172.21.15.88:0/2495626359 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4780b52c0 con 0x7fb47815cf40 2024-04-26T20:00:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.383+0000 7fb476800640 1 --2- 172.21.15.88:0/2495626359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815cf40 0x7fb47806dfe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb478076220 tx=0x7fb47002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440bef5dc6f155ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.383+0000 7fb474a00640 1 -- 172.21.15.88:0/2495626359 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470036070 con 0x7fb47815cf40 2024-04-26T20:00:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.383+0000 7fb474a00640 1 -- 172.21.15.88:0/2495626359 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb470031600 con 0x7fb47815cf40 2024-04-26T20:00:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.383+0000 7fb474a00640 1 -- 172.21.15.88:0/2495626359 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700319d0 con 0x7fb47815cf40 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 -- 172.21.15.88:0/2495626359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815cf40 msgr2=0x7fb47806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 --2- 172.21.15.88:0/2495626359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815cf40 0x7fb47806dfe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb478076220 tx=0x7fb47002a660 comp rx=0 tx=0).stop 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 -- 172.21.15.88:0/2495626359 shutdown_connections 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 --2- 172.21.15.88:0/2495626359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815cf40 0x7fb47806dfe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 --2- 172.21.15.88:0/2495626359 >> v2:172.21.15.132:3300/0 conn(0x7fb47815ee60 0x7fb4780b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 -- 172.21.15.88:0/2495626359 >> 172.21.15.88:0/2495626359 conn(0x7fb4780b2900 msgr2=0x7fb4780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 -- 172.21.15.88:0/2495626359 shutdown_connections 2024-04-26T20:00:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.384+0000 7fb477200640 1 -- 172.21.15.88:0/2495626359 wait complete. 2024-04-26T20:00:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.385+0000 7fb477200640 1 Processor -- start 2024-04-26T20:00:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.386+0000 7fb477200640 1 -- start start 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.386+0000 7fb477200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 0x7fb478157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.386+0000 7fb477200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb47815cf40 0x7fb4780cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.386+0000 7fb477200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb477200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4780b5a00 con 0x7fb47815ee60 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb477200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb4780b5880 con 0x7fb47815cf40 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb477200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb478165a00 con 0x7fb47815c450 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34020/0 (socket says 172.21.15.88:34020) 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 -- 172.21.15.88:0/1056858631 learned_addr learned my addr 172.21.15.88:0/1056858631 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb475400640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb47815cf40 0x7fb4780cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb475e00640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 0x7fb478157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 -- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 msgr2=0x7fb478157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 0x7fb478157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 -- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb47815cf40 msgr2=0x7fb4780cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb47815cf40 0x7fb4780cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb476800640 1 -- 172.21.15.88:0/1056858631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4781585a0 con 0x7fb47815ee60 2024-04-26T20:00:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.387+0000 7fb475e00640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 0x7fb478157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb476800640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb470009f40 tx=0x7fb470004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470036050 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb470042070 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb478158880 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470009510 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.388+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb478158e40 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.389+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4780693c0 con 0x7fb47815ee60 2024-04-26T20:00:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.390+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb470007bf0 con 0x7fb47815ee60 2024-04-26T20:00:01.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.391+0000 7fb46e000640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 0x7fb45806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.391+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4700acac0 con 0x7fb47815ee60 2024-04-26T20:00:01.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.391+0000 7fb475e00640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 0x7fb45806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.392+0000 7fb475e00640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 0x7fb45806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb468003eb0 tx=0x7fb468009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.395+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb47007aee0 con 0x7fb47815ee60 2024-04-26T20:00:01.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.593+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4780cf610 con 0x7fb47815ee60 2024-04-26T20:00:01.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.594+0000 7fb46e000640 1 -- 172.21.15.88:0/1056858631 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb47007a7e0 con 0x7fb47815ee60 2024-04-26T20:00:01.595 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 msgr2=0x7fb45806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 0x7fb45806edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb468003eb0 tx=0x7fb468009040 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 msgr2=0x7fb4781551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb470009f40 tx=0x7fb470004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 shutdown_connections 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb47815ee60 0x7fb4781551b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb47815cf40 0x7fb4780cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb45806c8f0 0x7fb45806edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 --2- 172.21.15.88:0/1056858631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb47815c450 0x7fb478157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 >> 172.21.15.88:0/1056858631 conn(0x7fb4780b2900 msgr2=0x7fb4780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 shutdown_connections 2024-04-26T20:00:01.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.597+0000 7fb477200640 1 -- 172.21.15.88:0/1056858631 wait complete. 2024-04-26T20:00:01.617 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:01.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.821+0000 7ff243000640 1 Processor -- start 2024-04-26T20:00:01.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.821+0000 7ff243000640 1 -- start start 2024-04-26T20:00:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff23c1467a0 0x7ff23c15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 0x7ff23c0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0d12a0 0x7ff23c06de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c146b80 con 0x7ff23c1467a0 2024-04-26T20:00:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c161730 con 0x7ff23c0b5ca0 2024-04-26T20:00:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.822+0000 7ff243000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0d16d0 con 0x7ff23c0d12a0 2024-04-26T20:00:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff23c1467a0 0x7ff23c15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff23c1467a0 0x7ff23c15dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34028/0 (socket says 172.21.15.88:34028) 2024-04-26T20:00:01.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 -- 172.21.15.88:0/3263430116 learned_addr learned my addr 172.21.15.88:0/3263430116 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241200640 1 --2- 172.21.15.88:0/3263430116 >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 0x7ff23c0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff242600640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0d12a0 0x7ff23c06de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 -- 172.21.15.88:0/3263430116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0d12a0 msgr2=0x7ff23c06de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0d12a0 0x7ff23c06de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 -- 172.21.15.88:0/3263430116 >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 msgr2=0x7ff23c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 --2- 172.21.15.88:0/3263430116 >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 0x7ff23c0d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.823+0000 7ff241c00640 1 -- 172.21.15.88:0/3263430116 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c15e3e0 con 0x7ff23c1467a0 2024-04-26T20:00:01.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.824+0000 7ff241200640 1 --2- 172.21.15.88:0/3263430116 >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 0x7ff23c0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.825+0000 7ff241c00640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c1467a0 0x7ff23c15dea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff23c160950 tx=0x7ff23002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358a5a8e4655d2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.825+0000 7ff23be00640 1 -- 172.21.15.88:0/3263430116 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230037070 con 0x7ff23c1467a0 2024-04-26T20:00:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.825+0000 7ff23be00640 1 -- 172.21.15.88:0/3263430116 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff230032440 con 0x7ff23c1467a0 2024-04-26T20:00:01.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.825+0000 7ff23be00640 1 -- 172.21.15.88:0/3263430116 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230032810 con 0x7ff23c1467a0 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 -- 172.21.15.88:0/3263430116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c1467a0 msgr2=0x7ff23c15dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c1467a0 0x7ff23c15dea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff23c160950 tx=0x7ff23002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 -- 172.21.15.88:0/3263430116 shutdown_connections 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0d12a0 0x7ff23c06de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 --2- 172.21.15.88:0/3263430116 >> v2:172.21.15.132:3300/0 conn(0x7ff23c0b5ca0 0x7ff23c0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 --2- 172.21.15.88:0/3263430116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c1467a0 0x7ff23c15dea0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 -- 172.21.15.88:0/3263430116 >> 172.21.15.88:0/3263430116 conn(0x7ff23c06c530 msgr2=0x7ff23c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.826+0000 7ff243000640 1 -- 172.21.15.88:0/3263430116 shutdown_connections 2024-04-26T20:00:01.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.827+0000 7ff243000640 1 -- 172.21.15.88:0/3263430116 wait complete. 2024-04-26T20:00:01.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.828+0000 7ff243000640 1 Processor -- start 2024-04-26T20:00:01.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.828+0000 7ff243000640 1 -- start start 2024-04-26T20:00:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0b5ca0 0x7ff23c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0d12a0 0x7ff23c144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1610e0 con 0x7ff23c145050 2024-04-26T20:00:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c160e00 con 0x7ff23c0d12a0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.829+0000 7ff243000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c1613c0 con 0x7ff23c0b5ca0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34030/0 (socket says 172.21.15.88:34030) 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 -- 172.21.15.88:0/416377320 learned_addr learned my addr 172.21.15.88:0/416377320 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff241c00640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0b5ca0 0x7ff23c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff241200640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0d12a0 0x7ff23c144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 -- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0b5ca0 msgr2=0x7ff23c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0b5ca0 0x7ff23c15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 -- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0d12a0 msgr2=0x7ff23c144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0d12a0 0x7ff23c144b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 -- 172.21.15.88:0/416377320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c15ccf0 con 0x7ff23c145050 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.830+0000 7ff242600640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff234007ec0 tx=0x7ff23400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.831+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234014420 con 0x7ff23c145050 2024-04-26T20:00:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.831+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff234019070 con 0x7ff23c145050 2024-04-26T20:00:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.831+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234014820 con 0x7ff23c145050 2024-04-26T20:00:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.831+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c073890 con 0x7ff23c145050 2024-04-26T20:00:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.831+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c142d20 con 0x7ff23c145050 2024-04-26T20:00:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.832+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c0693c0 con 0x7ff23c145050 2024-04-26T20:00:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.833+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2340149c0 con 0x7ff23c145050 2024-04-26T20:00:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.834+0000 7ff239600640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.834+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff23408c2b0 con 0x7ff23c145050 2024-04-26T20:00:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.834+0000 7ff241c00640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.835+0000 7ff241c00640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff230034000 tx=0x7ff23002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:01.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:01.839+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff23c15ccf0 con 0x7ff23c145050 2024-04-26T20:00:02.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.042+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff23c06f7a0 con 0x7ff23c145050 2024-04-26T20:00:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.043+0000 7ff239600640 1 -- 172.21.15.88:0/416377320 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff23405a710 con 0x7ff23c145050 2024-04-26T20:00:02.044 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 msgr2=0x7ff21c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 0x7ff21c06ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff230034000 tx=0x7ff23002e000 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 msgr2=0x7ff23c072b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff234007ec0 tx=0x7ff23400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 shutdown_connections 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff23c145050 0x7ff23c072b20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff23c0d12a0 0x7ff23c144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff21c06c920 0x7ff21c06ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 --2- 172.21.15.88:0/416377320 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff23c0b5ca0 0x7ff23c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 >> 172.21.15.88:0/416377320 conn(0x7ff23c06c530 msgr2=0x7ff23c0ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 shutdown_connections 2024-04-26T20:00:02.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.046+0000 7ff243000640 1 -- 172.21.15.88:0/416377320 wait complete. 2024-04-26T20:00:02.065 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.255+0000 7fc7aea00640 1 Processor -- start 2024-04-26T20:00:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.255+0000 7fc7aea00640 1 -- start start 2024-04-26T20:00:02.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7aea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc7a815f420 0x7fc7a80d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7aea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 0x7fc7a80c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7aea00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a815f800 con 0x7fc7a80d3680 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7aea00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a80d2c70 con 0x7fc7a815f420 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7aea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a80d3480 con 0x7fc7a80c9520 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc7a7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc7a815f420 0x7fc7a80d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc79f400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc7a80d3680 0x7fc7a80d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49856/0 (socket says 172.21.15.88:49856) 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.257+0000 7fc79f400640 1 -- 172.21.15.88:0/3217693754 learned_addr learned my addr 172.21.15.88:0/3217693754 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7aca00640 1 --2- 172.21.15.88:0/3217693754 >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 0x7fc7a80c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7a7e00640 1 -- 172.21.15.88:0/3217693754 >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 msgr2=0x7fc7a80c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7a7e00640 1 --2- 172.21.15.88:0/3217693754 >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 0x7fc7a80c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7a7e00640 1 -- 172.21.15.88:0/3217693754 >> v1:172.21.15.88:6789/0 conn(0x7fc7a80d3680 legacy=0x7fc7a80d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7a7e00640 1 -- 172.21.15.88:0/3217693754 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a80d2ec0 con 0x7fc7a815f420 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7a7400640 1 -- 172.21.15.88:0/3217693754 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678029680 0 0) 0x7fc7a815f800 con 0x7fc7a80d3680 2024-04-26T20:00:02.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.258+0000 7fc7aca00640 1 --2- 172.21.15.88:0/3217693754 >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 0x7fc7a80c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:02.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.259+0000 7fc7a7e00640 1 --2- 172.21.15.88:0/3217693754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a815f420 0x7fc7a80d5900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80c8940 tx=0x7fc79802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420a0cd4bb79bd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.259+0000 7fc7a7400640 1 -- 172.21.15.88:0/3217693754 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798037070 con 0x7fc7a815f420 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.259+0000 7fc7a7400640 1 -- 172.21.15.88:0/3217693754 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798032440 con 0x7fc7a815f420 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.259+0000 7fc7a7400640 1 -- 172.21.15.88:0/3217693754 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79802ae60 con 0x7fc7a815f420 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 -- 172.21.15.88:0/3217693754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a815f420 msgr2=0x7fc7a80d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3217693754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a815f420 0x7fc7a80d5900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80c8940 tx=0x7fc79802a110 comp rx=0 tx=0).stop 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 -- 172.21.15.88:0/3217693754 shutdown_connections 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3217693754 >> v2:172.21.15.132:3300/0 conn(0x7fc7a80c9520 0x7fc7a80c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3217693754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a815f420 0x7fc7a80d5900 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 -- 172.21.15.88:0/3217693754 >> 172.21.15.88:0/3217693754 conn(0x7fc7a809e3e0 msgr2=0x7fc7a80d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 -- 172.21.15.88:0/3217693754 shutdown_connections 2024-04-26T20:00:02.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.260+0000 7fc7aea00640 1 -- 172.21.15.88:0/3217693754 wait complete. 2024-04-26T20:00:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.261+0000 7fc7aea00640 1 Processor -- start 2024-04-26T20:00:02.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.262+0000 7fc7aea00640 1 -- start start 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a80c9520 0x7fc7a80b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 0x7fc7a8161f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a80c9000 con 0x7fc7a80d3680 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a80c8d20 con 0x7fc7a80c9520 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a80d0f30 con 0x7fc7a815f420 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34050/0 (socket says 172.21.15.88:34050) 2024-04-26T20:00:02.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 -- 172.21.15.88:0/3637181677 learned_addr learned my addr 172.21.15.88:0/3637181677 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7a7e00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a80c9520 0x7fc7a80b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aca00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 0x7fc7a8161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 -- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 msgr2=0x7fc7a8161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 0x7fc7a8161f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 -- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a80c9520 msgr2=0x7fc7a80b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a80c9520 0x7fc7a80b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc79f400640 1 -- 172.21.15.88:0/3637181677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a814d9e0 con 0x7fc7a80d3680 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.263+0000 7fc7aca00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 0x7fc7a8161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:02.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc79f400640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c4d0 tx=0x7fc79000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790018070 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc790007500 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a8161c60 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc790004a20 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.264+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a81f8650 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.265+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a8069540 con 0x7fc7a80d3680 2024-04-26T20:00:02.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.266+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc79001b070 con 0x7fc7a80d3680 2024-04-26T20:00:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.267+0000 7fc7a4c00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 0x7fc78406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.267+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc79008aee0 con 0x7fc7a80d3680 2024-04-26T20:00:02.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.267+0000 7fc7a7e00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 0x7fc78406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.268+0000 7fc7a7e00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 0x7fc78406ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc798034000 tx=0x7fc79802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.271+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc790059270 con 0x7fc7a80d3680 2024-04-26T20:00:02.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.495+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7a80d15b0 con 0x7fc7a80d3680 2024-04-26T20:00:02.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.497+0000 7fc7a4c00640 1 -- 172.21.15.88:0/3637181677 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7fc790058b70 con 0x7fc7a80d3680 2024-04-26T20:00:02.497 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:02.498 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 msgr2=0x7fc78406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 0x7fc78406ede0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc798034000 tx=0x7fc79802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 msgr2=0x7fc7a814d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc79000c4d0 tx=0x7fc79000c990 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 shutdown_connections 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc78406c920 0x7fc78406ede0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc7a815f420 0x7fc7a8161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a80d3680 0x7fc7a814d4a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 --2- 172.21.15.88:0/3637181677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a80c9520 0x7fc7a80b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.499+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 >> 172.21.15.88:0/3637181677 conn(0x7fc7a809e3e0 msgr2=0x7fc7a80c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.500+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 shutdown_connections 2024-04-26T20:00:02.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.500+0000 7fc7aea00640 1 -- 172.21.15.88:0/3637181677 wait complete. 2024-04-26T20:00:02.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.714+0000 7f4692600640 1 Processor -- start 2024-04-26T20:00:02.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.714+0000 7f4692600640 1 -- start start 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.715+0000 7f4692600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f468c15f450 0x7f468c0d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f4692600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c0d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f4692600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468c0cfd60 con 0x7f468c0d3bd0 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f4692600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f468c0b5520 con 0x7f468c15f450 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f4692600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f468c0d4000 con 0x7f468c0d26a0 2024-04-26T20:00:02.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f468c15f450 0x7f468c0d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468be00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f468c0d3bd0 0x7f468c163000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49872/0 (socket says 172.21.15.88:49872) 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468be00640 1 -- 172.21.15.88:0/892917765 learned_addr learned my addr 172.21.15.88:0/892917765 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f4683e00640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c0d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468b400640 1 -- 172.21.15.88:0/892917765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 msgr2=0x7f468c0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468b400640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c0d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468b400640 1 -- 172.21.15.88:0/892917765 >> v1:172.21.15.88:6789/0 conn(0x7f468c0d3bd0 legacy=0x7f468c163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468b400640 1 -- 172.21.15.88:0/892917765 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c0d0040 con 0x7f468c15f450 2024-04-26T20:00:02.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.716+0000 7f468aa00640 1 -- 172.21.15.88:0/892917765 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444306925 0 0) 0x7f468c0cfd60 con 0x7f468c0d3bd0 2024-04-26T20:00:02.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.717+0000 7f468b400640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c15f450 0x7f468c0d2160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097a0 tx=0x7f467c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8e65bb333ac8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f468aa00640 1 -- 172.21.15.88:0/892917765 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c037070 con 0x7f468c15f450 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f468aa00640 1 -- 172.21.15.88:0/892917765 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c032440 con 0x7f468c15f450 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f468aa00640 1 -- 172.21.15.88:0/892917765 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c032810 con 0x7f468c15f450 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 -- 172.21.15.88:0/892917765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c15f450 msgr2=0x7f468c0d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c15f450 0x7f468c0d2160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f467c0097a0 tx=0x7f467c02a180 comp rx=0 tx=0).stop 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 -- 172.21.15.88:0/892917765 shutdown_connections 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c0d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 --2- 172.21.15.88:0/892917765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c15f450 0x7f468c0d2160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 -- 172.21.15.88:0/892917765 >> 172.21.15.88:0/892917765 conn(0x7f468c0d50b0 msgr2=0x7f468c0d54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 -- 172.21.15.88:0/892917765 shutdown_connections 2024-04-26T20:00:02.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.718+0000 7f4692600640 1 -- 172.21.15.88:0/892917765 wait complete. 2024-04-26T20:00:02.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.720+0000 7f4692600640 1 Processor -- start 2024-04-26T20:00:02.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.720+0000 7f4692600640 1 -- start start 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 0x7f468c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f468c0d08e0 con 0x7f468c0d3bd0 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f468c0d0600 con 0x7f468c15f450 2024-04-26T20:00:02.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4692600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f468c15f830 con 0x7f468c0d26a0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.721+0000 7f4683e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34078/0 (socket says 172.21.15.88:34078) 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 -- 172.21.15.88:0/1537697213 learned_addr learned my addr 172.21.15.88:0/1537697213 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f468b400640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 -- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 msgr2=0x7f468c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f468be00640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 0x7f468c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 -- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 msgr2=0x7f468c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 0x7f468c06df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 -- 172.21.15.88:0/1537697213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c0ce070 con 0x7f468c0d3bd0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f468be00640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 0x7f468c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.722+0000 7f4683e00640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f467400d830 tx=0x7f467400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.723+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400b6c0 con 0x7f468c0d3bd0 2024-04-26T20:00:02.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.723+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4674019070 con 0x7f468c0d3bd0 2024-04-26T20:00:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.723+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400ba80 con 0x7f468c0d3bd0 2024-04-26T20:00:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.723+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c0cc220 con 0x7f468c0d3bd0 2024-04-26T20:00:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.723+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c0cc7e0 con 0x7f468c0d3bd0 2024-04-26T20:00:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.724+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c069540 con 0x7f468c0d3bd0 2024-04-26T20:00:02.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.725+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f467401e460 con 0x7f468c0d3bd0 2024-04-26T20:00:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.726+0000 7f4683400640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 0x7f466806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.726+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f467408c240 con 0x7f468c0d3bd0 2024-04-26T20:00:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.726+0000 7f468b400640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 0x7f466806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:02.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.727+0000 7f468b400640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 0x7f466806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f467c034000 tx=0x7f467c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:02.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.734+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f467405a660 con 0x7f468c0d3bd0 2024-04-26T20:00:02.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.939+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f468c06ecd0 con 0x7f468c0d3bd0 2024-04-26T20:00:02.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.941+0000 7f4683400640 1 -- 172.21.15.88:0/1537697213 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f4674059f60 con 0x7f468c0d3bd0 2024-04-26T20:00:02.942 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:02.942 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 msgr2=0x7f466806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 0x7f466806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f467c034000 tx=0x7f467c02e000 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 msgr2=0x7f468c0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f467400d830 tx=0x7f467400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 shutdown_connections 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f468c15f450 0x7f468c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f468c0d3bd0 0x7f468c0cd5f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f466806c8f0 0x7f466806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 --2- 172.21.15.88:0/1537697213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f468c0d26a0 0x7f468c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:02.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 >> 172.21.15.88:0/1537697213 conn(0x7f468c0d50b0 msgr2=0x7f468c0b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:02.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 shutdown_connections 2024-04-26T20:00:02.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:02.944+0000 7f4692600640 1 -- 172.21.15.88:0/1537697213 wait complete. 2024-04-26T20:00:02.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.137+0000 7f5726000640 1 Processor -- start 2024-04-26T20:00:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.137+0000 7f5726000640 1 -- start start 2024-04-26T20:00:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5726000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57200d12b0 0x7f57200b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5726000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5720146a60 con 0x7f57200d0e70 2024-04-26T20:00:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5726000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f572006dd60 con 0x7f5720146680 2024-04-26T20:00:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5726000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f572006e970 con 0x7f57200d12b0 2024-04-26T20:00:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5725600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57200d12b0 0x7f57200b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5725600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57200d12b0 0x7f57200b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34084/0 (socket says 172.21.15.88:34084) 2024-04-26T20:00:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5725600640 1 -- 172.21.15.88:0/3765441411 learned_addr learned my addr 172.21.15.88:0/3765441411 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:03.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f571fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f57200d0e70 0x7f57200b3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49878/0 (socket says 172.21.15.88:49878) 2024-04-26T20:00:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5725600640 1 -- 172.21.15.88:0/3765441411 >> v1:172.21.15.132:6789/0 conn(0x7f5720146680 legacy=0x7f57200b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f5725600640 1 -- 172.21.15.88:0/3765441411 >> v1:172.21.15.88:6789/0 conn(0x7f57200d0e70 legacy=0x7f57200b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.140+0000 7f5725600640 1 -- 172.21.15.88:0/3765441411 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572006e040 con 0x7f57200d12b0 2024-04-26T20:00:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.139+0000 7f571f400640 1 -- 172.21.15.88:0/3765441411 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643413316 0 0) 0x7f5720146a60 con 0x7f57200d0e70 2024-04-26T20:00:03.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.140+0000 7f5725600640 1 --2- 172.21.15.88:0/3765441411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d12b0 0x7f57200b4230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5718009e50 tx=0x7f571802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca6755b562f215c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:03.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f571f400640 1 -- 172.21.15.88:0/3765441411 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5718036070 con 0x7f57200d12b0 2024-04-26T20:00:03.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f571f400640 1 -- 172.21.15.88:0/3765441411 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5718031600 con 0x7f57200d12b0 2024-04-26T20:00:03.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f571f400640 1 -- 172.21.15.88:0/3765441411 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57180319f0 con 0x7f57200d12b0 2024-04-26T20:00:03.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 -- 172.21.15.88:0/3765441411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d12b0 msgr2=0x7f57200b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 --2- 172.21.15.88:0/3765441411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d12b0 0x7f57200b4230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5718009e50 tx=0x7f571802a7c0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 -- 172.21.15.88:0/3765441411 shutdown_connections 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 --2- 172.21.15.88:0/3765441411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d12b0 0x7f57200b4230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 -- 172.21.15.88:0/3765441411 >> 172.21.15.88:0/3765441411 conn(0x7f572006c740 msgr2=0x7f572006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 -- 172.21.15.88:0/3765441411 shutdown_connections 2024-04-26T20:00:03.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.141+0000 7f5726000640 1 -- 172.21.15.88:0/3765441411 wait complete. 2024-04-26T20:00:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.143+0000 7f5726000640 1 Processor -- start 2024-04-26T20:00:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.143+0000 7f5726000640 1 -- start start 2024-04-26T20:00:03.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57200d12b0 0x7f57200c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5720146680 0x7f5720141270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f572006e600 con 0x7f57200d0e70 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f57200b60a0 con 0x7f5720146680 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5726000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f57200d1700 con 0x7f57200d12b0 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5724c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5724c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34092/0 (socket says 172.21.15.88:34092) 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.144+0000 7f5724c00640 1 -- 172.21.15.88:0/2263372371 learned_addr learned my addr 172.21.15.88:0/2263372371 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 -- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57200d12b0 msgr2=0x7f57200c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57200d12b0 0x7f57200c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 -- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5720146680 msgr2=0x7f5720141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5720146680 0x7f5720141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 -- 172.21.15.88:0/2263372371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57200c6390 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5724c00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f571400c430 tx=0x7f571400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714017070 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57140052d0 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5714003830 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57200c6670 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.145+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57201f2510 con 0x7f57200d0e70 2024-04-26T20:00:03.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.146+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57200693c0 con 0x7f57200d0e70 2024-04-26T20:00:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.147+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f571401a070 con 0x7f57200d0e70 2024-04-26T20:00:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.148+0000 7f571cc00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 0x7f570006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.148+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f571408af60 con 0x7f57200d0e70 2024-04-26T20:00:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.148+0000 7f571fe00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 0x7f570006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:03.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.149+0000 7f571fe00640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 0x7f570006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f570c007fd0 tx=0x7f570c007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:03.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.153+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57140592f0 con 0x7f57200d0e70 2024-04-26T20:00:03.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.384+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57200cf330 con 0x7f57200d0e70 2024-04-26T20:00:03.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.385+0000 7f571cc00640 1 -- 172.21.15.88:0/2263372371 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f5714058bf0 con 0x7f57200d0e70 2024-04-26T20:00:03.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:03.385 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 msgr2=0x7f570006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 0x7f570006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f570c007fd0 tx=0x7f570c007900 comp rx=0 tx=0).stop 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 msgr2=0x7f57201f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f571400c430 tx=0x7f571400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 shutdown_connections 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5720146680 0x7f5720141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f570006c920 0x7f570006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57200d12b0 0x7f57200c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 --2- 172.21.15.88:0/2263372371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57200d0e70 0x7f57201f1fd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 >> 172.21.15.88:0/2263372371 conn(0x7f572006c740 msgr2=0x7f572015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.387+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 shutdown_connections 2024-04-26T20:00:03.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:03.388+0000 7f5726000640 1 -- 172.21.15.88:0/2263372371 wait complete. 2024-04-26T20:00:03.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T19:59:45.722094+0000', 'modified': '2024-04-26T19:59:48.769966+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6837/3248668459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6835', 'nonce': 3248668459}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 3248668459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-26T20:00:03.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:00:03.809 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:00:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.016+0000 7f8363200640 1 Processor -- start 2024-04-26T20:00:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.016+0000 7f8363200640 1 -- start start 2024-04-26T20:00:04.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.017+0000 7f8363200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f83640d0e70 0x7f83640d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8363200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f83640b3b20 0x7f83640b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8363200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8364146a60 con 0x7f8364146680 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8363200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f836406e970 con 0x7f83640b3b20 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8363200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f83640b60a0 con 0x7f83640d0e70 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8361e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8364146680 0x7f83640b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:49894/0 (socket says 172.21.15.88:49894) 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8361e00640 1 -- 172.21.15.88:0/2078898040 learned_addr learned my addr 172.21.15.88:0/2078898040 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8362800640 1 --2- 172.21.15.88:0/2078898040 >> v2:172.21.15.88:3300/0 conn(0x7f83640b3b20 0x7f83640b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8361400640 1 --2- 172.21.15.88:0/2078898040 >> v2:172.21.15.132:3300/0 conn(0x7f83640d0e70 0x7f83640d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8362800640 1 -- 172.21.15.88:0/2078898040 >> v2:172.21.15.132:3300/0 conn(0x7f83640d0e70 msgr2=0x7f83640d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8362800640 1 --2- 172.21.15.88:0/2078898040 >> v2:172.21.15.132:3300/0 conn(0x7f83640d0e70 0x7f83640d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8362800640 1 -- 172.21.15.88:0/2078898040 >> v1:172.21.15.88:6789/0 conn(0x7f8364146680 legacy=0x7f83640b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.018+0000 7f8362800640 1 -- 172.21.15.88:0/2078898040 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836406dd60 con 0x7f83640b3b20 2024-04-26T20:00:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.019+0000 7f8362800640 1 --2- 172.21.15.88:0/2078898040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640b3b20 0x7f83640b3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8364160890 tx=0x7f835c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7324d36523ae21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8360a00640 1 -- 172.21.15.88:0/2078898040 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c007740 con 0x7f83640b3b20 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8360a00640 1 -- 172.21.15.88:0/2078898040 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c0078e0 con 0x7f83640b3b20 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8360a00640 1 -- 172.21.15.88:0/2078898040 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c007cb0 con 0x7f83640b3b20 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 -- 172.21.15.88:0/2078898040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640b3b20 msgr2=0x7f83640b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 --2- 172.21.15.88:0/2078898040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640b3b20 0x7f83640b3f70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8364160890 tx=0x7f835c02a680 comp rx=0 tx=0).stop 2024-04-26T20:00:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 -- 172.21.15.88:0/2078898040 shutdown_connections 2024-04-26T20:00:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 --2- 172.21.15.88:0/2078898040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640b3b20 0x7f83640b3f70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 --2- 172.21.15.88:0/2078898040 >> v2:172.21.15.132:3300/0 conn(0x7f83640d0e70 0x7f83640d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 -- 172.21.15.88:0/2078898040 >> 172.21.15.88:0/2078898040 conn(0x7f836406c740 msgr2=0x7f836406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 -- 172.21.15.88:0/2078898040 shutdown_connections 2024-04-26T20:00:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.020+0000 7f8363200640 1 -- 172.21.15.88:0/2078898040 wait complete. 2024-04-26T20:00:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.022+0000 7f8363200640 1 Processor -- start 2024-04-26T20:00:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.022+0000 7f8363200640 1 -- start start 2024-04-26T20:00:04.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83640b3b20 0x7f836415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8364146680 0x7f8364133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f836406e320 con 0x7f83640d0e70 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f83640d17f0 con 0x7f83640b3b20 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8363200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f836406e600 con 0x7f8364146680 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.023+0000 7f8361400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34100/0 (socket says 172.21.15.88:34100) 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 -- 172.21.15.88:0/1747219393 learned_addr learned my addr 172.21.15.88:0/1747219393 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 -- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8364146680 msgr2=0x7f8364133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8364146680 0x7f8364133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 -- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83640b3b20 msgr2=0x7f836415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83640b3b20 0x7f836415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 -- 172.21.15.88:0/1747219393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8364144ec0 con 0x7f83640d0e70 2024-04-26T20:00:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f8361400640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f835000d800 tx=0x7f835000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f835a000640 1 -- 172.21.15.88:0/1747219393 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000b6c0 con 0x7f83640d0e70 2024-04-26T20:00:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.024+0000 7f835a000640 1 -- 172.21.15.88:0/1747219393 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8350019070 con 0x7f83640d0e70 2024-04-26T20:00:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.025+0000 7f835a000640 1 -- 172.21.15.88:0/1747219393 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835000ba80 con 0x7f83640d0e70 2024-04-26T20:00:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.025+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83641451a0 con 0x7f83640d0e70 2024-04-26T20:00:04.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.025+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83641353e0 con 0x7f83640d0e70 2024-04-26T20:00:04.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.027+0000 7f835a000640 1 -- 172.21.15.88:0/1747219393 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f835000bc20 con 0x7f83640d0e70 2024-04-26T20:00:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.028+0000 7f835a000640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.028+0000 7f835a000640 1 -- 172.21.15.88:0/1747219393 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f835008bff0 con 0x7f83640d0e70 2024-04-26T20:00:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.028+0000 7f8361e00640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.028+0000 7f836b9f9740 1 Processor -- start 2024-04-26T20:00:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.029+0000 7f8361e00640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 0x7f834406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354007a80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.029+0000 7f836b9f9740 1 -- start start 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.029+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489c97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.029+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cd1e80 0x557489c95b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.029+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f836b9f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557489bf9600 con 0x557489cc0270 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557489b96100 con 0x557489cd1e80 2024-04-26T20:00:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x557489bf55c0 con 0x557489c957b0 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34112/0 (socket says 172.21.15.88:34112) 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 -- 172.21.15.88:0/3824908717 learned_addr learned my addr 172.21.15.88:0/3824908717 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8361e00640 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489c97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 -- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 msgr2=0x557489c97910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489c97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8361400640 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cd1e80 0x557489c95b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 -- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cd1e80 msgr2=0x557489c95b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cd1e80 0x557489c95b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 -- 172.21.15.88:0/3824908717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748990b2d0 con 0x557489cc0270 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.030+0000 7f8362800640 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f835c009e50 tx=0x7f835c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa492e4dcc11b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f834e600640 1 -- 172.21.15.88:0/3824908717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c040070 con 0x557489cc0270 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f834e600640 1 -- 172.21.15.88:0/3824908717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c007740 con 0x557489cc0270 2024-04-26T20:00:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f834e600640 1 -- 172.21.15.88:0/3824908717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c007b30 con 0x557489cc0270 2024-04-26T20:00:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 -- 172.21.15.88:0/3824908717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 msgr2=0x557489cc06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f835c009e50 tx=0x7f835c02c000 comp rx=0 tx=0).stop 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 -- 172.21.15.88:0/3824908717 shutdown_connections 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cc0270 0x557489cc06f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cd1e80 0x557489c95b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3824908717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489c97910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.031+0000 7f836b9f9740 1 -- 172.21.15.88:0/3824908717 >> 172.21.15.88:0/3824908717 conn(0x557489c29d10 msgr2=0x557489cdb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.032+0000 7f836b9f9740 1 -- 172.21.15.88:0/3824908717 shutdown_connections 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.032+0000 7f836b9f9740 1 -- 172.21.15.88:0/3824908717 wait complete. 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.032+0000 7f836b9f9740 1 -- start start 2024-04-26T20:00:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.032+0000 7f836b9f9740 1 Processor -- start 2024-04-26T20:00:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cbd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cd35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557489b80f70 con 0x557489cd1e80 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557489cbddc0 con 0x557489cc0270 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x557489cebe80 con 0x557489c957b0 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.033+0000 7f836b9f9740 10 client.0 _ll_register_callbacks cb 0x557489bef480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34116/0 (socket says 172.21.15.88:34116) 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 -- 172.21.15.88:0/3120196850 learned_addr learned my addr 172.21.15.88:0/3120196850 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 -- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 msgr2=0x557489cbd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cbd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 -- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 msgr2=0x557489cd35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cd35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 -- 172.21.15.88:0/3120196850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574899d56b0 con 0x557489cd1e80 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8362800640 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f835c041000 tx=0x7f835c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d98bec217c6c6dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8342a00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.034+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c040040 con 0x557489cd1e80 2024-04-26T20:00:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f835c04faf0 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c007740 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f835c003e40 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 10 client.4617 fetch_fsmap learned FSMap version 7 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557489cc11d0 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c007740 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f835c059080 con 0x557489cd1e80 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 10 client.4617 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 10 client.4617 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:00:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 20 client.4617 populate_metadata read hostname 'smithi088' 2024-04-26T20:00:04.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cd24c0 0x557489cd28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 4 client.4617 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:00:04.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.035+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 --> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557489c07210 con 0x557489cd24c0 2024-04-26T20:00:04.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.036+0000 7f8361e00640 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cd24c0 0x557489cd28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.036+0000 7f8342a00640 10 client.4617 ms_handle_connect on v2:172.21.15.88:6835/3248668459 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.036+0000 7f8361e00640 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cd24c0 0x557489cd28a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418e7c36dcc0d38 server_cookie=82068678ecb4d93f in_seq=0 out_seq=0 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f8342a00640 1 -- 172.21.15.88:0/3120196850 <== mds.0 v2:172.21.15.88:6835/3248668459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557489c07210 con 0x557489cd24c0 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f8342a00640 10 client.4617 handle_command_reply: tid=0 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f836b9f9740 1 client.4617 shutdown 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 msgr2=0x557489caf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f835c041000 tx=0x7f835c004ab0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 shutdown_connections 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.037+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cd24c0 0x557489cd28a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489caf470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cd35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3120196850 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cbd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f8342a00640 0 client.4617 ms_handle_reset on v2:172.21.15.88:6835/3248668459 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 >> 172.21.15.88:0/3120196850 conn(0x557489c29d10 msgr2=0x557489cde3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 shutdown_connections 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 1 -- 172.21.15.88:0/3120196850 wait complete. 2024-04-26T20:00:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.038+0000 7f836b9f9740 20 client.4617 trim_cache size 0 max 16384 2024-04-26T20:00:04.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.057+0000 7f836b9f9740 1 Processor -- start 2024-04-26T20:00:04.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.057+0000 7f836b9f9740 1 -- start start 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83642004b0 con 0x557489cd1e80 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f83641f8f70 con 0x557489cc0270 2024-04-26T20:00:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.058+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f83641366e0 con 0x557489c957b0 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34122/0 (socket says 172.21.15.88:34122) 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 -- 172.21.15.88:0/3388091941 learned_addr learned my addr 172.21.15.88:0/3388091941 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 -- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 msgr2=0x557489cc14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 -- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 msgr2=0x557489cc1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 -- 172.21.15.88:0/3388091941 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557489c8c680 con 0x557489cd1e80 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f8362800640 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f835c002230 tx=0x7f835c047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b38309ac1c456 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f834dc00640 1 -- 172.21.15.88:0/3388091941 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c040040 con 0x557489cd1e80 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f834dc00640 1 -- 172.21.15.88:0/3388091941 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c05c020 con 0x557489cd1e80 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.059+0000 7f834dc00640 1 -- 172.21.15.88:0/3388091941 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c003ca0 con 0x557489cd1e80 2024-04-26T20:00:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 -- 172.21.15.88:0/3388091941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 msgr2=0x557489cd0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f835c002230 tx=0x7f835c047040 comp rx=0 tx=0).stop 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 -- 172.21.15.88:0/3388091941 shutdown_connections 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489cd0c00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cc1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 --2- 172.21.15.88:0/3388091941 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489cc14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 -- 172.21.15.88:0/3388091941 >> 172.21.15.88:0/3388091941 conn(0x557489c29d10 msgr2=0x557489c96690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 -- 172.21.15.88:0/3388091941 shutdown_connections 2024-04-26T20:00:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.060+0000 7f836b9f9740 1 -- 172.21.15.88:0/3388091941 wait complete. 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.061+0000 7f836b9f9740 1 -- start start 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.061+0000 7f836b9f9740 1 Processor -- start 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489d58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cbab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557489cafe00 con 0x557489cd1e80 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x557489caf9b0 con 0x557489cc0270 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x557489ceb2f0 con 0x557489c957b0 2024-04-26T20:00:04.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f836b9f9740 10 client.0 _ll_register_callbacks cb 0x7f835c009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f8362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34136/0 (socket says 172.21.15.88:34136) 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.062+0000 7f8362800640 1 -- 172.21.15.88:0/2993682323 learned_addr learned my addr 172.21.15.88:0/2993682323 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8361e00640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489d58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8361400640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 -- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 msgr2=0x557489d58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489d58270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 -- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 msgr2=0x557489cbab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cbab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 -- 172.21.15.88:0/2993682323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557489cc39b0 con 0x557489cd1e80 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8362800640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f835c009920 tx=0x7f835c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9601cee54967a441 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8342a00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:00:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c040070 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f835c062b00 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.063+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c05c020 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f835c007b00 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 10 client.4623 fetch_fsmap learned FSMap version 7 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557489cea310 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c05c020 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f835c069050 con 0x557489cd1e80 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 10 client.4623 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 10 client.4623 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 20 client.4623 populate_metadata read hostname 'smithi088' 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cc14f0 0x557489cc18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 4 client.4623 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 --> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] -- command(tid 0: {"prefix": "status"}) -- 0x557489cc2c60 con 0x557489cc14f0 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.064+0000 7f8361e00640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cc14f0 0x557489cc18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f8361e00640 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cc14f0 0x557489cc18d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eecc44e38e55bfa server_cookie=9ec6c8de1e425a3a in_seq=0 out_seq=0 2024-04-26T20:00:04.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f8342a00640 10 client.4623 ms_handle_connect on v2:172.21.15.88:6835/3248668459 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f8342a00640 1 -- 172.21.15.88:0/2993682323 <== mds.0 v2:172.21.15.88:6835/3248668459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557489cc2c60 con 0x557489cc14f0 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f8342a00640 10 client.4623 handle_command_reply: tid=0 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 client.4623 shutdown 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 msgr2=0x557489ce9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f835c009920 tx=0x7f835c02e000 comp rx=0 tx=0).stop 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 shutdown_connections 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:6835/3248668459,v1:172.21.15.88:6837/3248668459] conn(0x557489cc14f0 0x557489cc18d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557489cd1e80 0x557489ce9dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557489cc0270 0x557489cbab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 --2- 172.21.15.88:0/2993682323 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x557489c957b0 0x557489d58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f8342a00640 0 client.4623 ms_handle_reset on v2:172.21.15.88:6835/3248668459 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 >> 172.21.15.88:0/2993682323 conn(0x557489c29d10 msgr2=0x557489cdb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 shutdown_connections 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.065+0000 7f836b9f9740 1 -- 172.21.15.88:0/2993682323 wait complete. 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f836b9f9740 20 client.4623 trim_cache size 0 max 16384 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "100f2996-174e-4373-bd56-473fe683ba7a", 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4270, 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-04-26T20:00:04.070 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 17.806998295, 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 7, 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 20, 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 20, 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 22.860942421000001 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:00:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 msgr2=0x7f834406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 0x7f834406edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354007a80 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 msgr2=0x7f8364144420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f835000d800 tx=0x7f835000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 shutdown_connections 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f834406c8f0 0x7f834406edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8364146680 0x7f8364133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83640d0e70 0x7f8364144420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 --2- 172.21.15.88:0/1747219393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83640b3b20 0x7f836415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 >> 172.21.15.88:0/1747219393 conn(0x7f836406c740 msgr2=0x7f8364140cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 shutdown_connections 2024-04-26T20:00:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.066+0000 7f8363200640 1 -- 172.21.15.88:0/1747219393 wait complete. 2024-04-26T20:00:04.085 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:00:04.094 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:00:04.094 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:00:04.094 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:00:04.095 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T20:00:04.116 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T20:00:04.116 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T20:00:04.116 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-26T20:00:04.116 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:04.117 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:04.117 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:04.117 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:00:04.117 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:04.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:04.117 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:00:04.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:04.176 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-26T20:00:04.265 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-04-26T20:00:04.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:00:04.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:04.270 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:00:04.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:04.326 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-04-26T20:00:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-04-26T20:00:04.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:00:04.415 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T20:00:04.415 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:00:04.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.616+0000 7f6667e00640 1 Processor -- start 2024-04-26T20:00:04.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.616+0000 7f6667e00640 1 -- start start 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6667e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f666815f1e0 0x7f66680b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6667e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f66680d5b60 0x7f66680d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6667e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666815f5c0 con 0x7f66680d2fc0 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6667e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f66680b8670 con 0x7f666815f1e0 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6667e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f66680b8950 con 0x7f66680d5b60 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f666815f1e0 0x7f66680b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f666815f1e0 0x7f66680b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34144/0 (socket says 172.21.15.88:34144) 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 -- 172.21.15.88:0/1166821931 learned_addr learned my addr 172.21.15.88:0/1166821931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f665fe00640 1 --2- 172.21.15.88:0/1166821931 >> v2:172.21.15.132:3300/0 conn(0x7f66680d5b60 0x7f66680d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 -- 172.21.15.88:0/1166821931 >> v2:172.21.15.132:3300/0 conn(0x7f66680d5b60 msgr2=0x7f66680d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 --2- 172.21.15.88:0/1166821931 >> v2:172.21.15.132:3300/0 conn(0x7f66680d5b60 0x7f66680d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 -- 172.21.15.88:0/1166821931 >> v1:172.21.15.132:6789/0 conn(0x7f66680d2fc0 legacy=0x7f66680cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.618+0000 7f6666a00640 1 -- 172.21.15.88:0/1166821931 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680b8c30 con 0x7f666815f1e0 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.619+0000 7f6666a00640 1 --2- 172.21.15.88:0/1166821931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666815f1e0 0x7f66680b7d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f666808db50 tx=0x7f665802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a5f16fce890ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6666000640 1 -- 172.21.15.88:0/1166821931 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658037070 con 0x7f666815f1e0 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6666000640 1 -- 172.21.15.88:0/1166821931 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6658032440 con 0x7f666815f1e0 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6666000640 1 -- 172.21.15.88:0/1166821931 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658032810 con 0x7f666815f1e0 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 -- 172.21.15.88:0/1166821931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666815f1e0 msgr2=0x7f66680b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 --2- 172.21.15.88:0/1166821931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666815f1e0 0x7f66680b7d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f666808db50 tx=0x7f665802a230 comp rx=0 tx=0).stop 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 -- 172.21.15.88:0/1166821931 shutdown_connections 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 --2- 172.21.15.88:0/1166821931 >> v2:172.21.15.132:3300/0 conn(0x7f66680d5b60 0x7f66680d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 --2- 172.21.15.88:0/1166821931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f666815f1e0 0x7f66680b7d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.620+0000 7f6667e00640 1 -- 172.21.15.88:0/1166821931 >> 172.21.15.88:0/1166821931 conn(0x7f666809e2d0 msgr2=0x7f66680d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.621+0000 7f6667e00640 1 -- 172.21.15.88:0/1166821931 shutdown_connections 2024-04-26T20:00:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.621+0000 7f6667e00640 1 -- 172.21.15.88:0/1166821931 wait complete. 2024-04-26T20:00:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.622+0000 7f6667e00640 1 Processor -- start 2024-04-26T20:00:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 -- start start 2024-04-26T20:00:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66680d2fc0 0x7f6668161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f666815f1e0 0x7f666814f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b82d0 con 0x7f66680d5b60 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b91f0 con 0x7f66680d2fc0 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.623+0000 7f6667e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f66680b8450 con 0x7f666815f1e0 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34158/0 (socket says 172.21.15.88:34158) 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 -- 172.21.15.88:0/632684419 learned_addr learned my addr 172.21.15.88:0/632684419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f6666a00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66680d2fc0 0x7f6668161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f6667400640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f666815f1e0 0x7f666814f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 -- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f666815f1e0 msgr2=0x7f666814f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f666815f1e0 0x7f666814f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 -- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66680d2fc0 msgr2=0x7f6668161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66680d2fc0 0x7f6668161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 -- 172.21.15.88:0/632684419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6668160430 con 0x7f66680d5b60 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.624+0000 7f665fe00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f665000d800 tx=0x7f665000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.625+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000b6c0 con 0x7f66680d5b60 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.625+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650019070 con 0x7f66680d5b60 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.625+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000ba80 con 0x7f66680d5b60 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.625+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6668160710 con 0x7f66680d5b60 2024-04-26T20:00:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.625+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6668160cd0 con 0x7f66680d5b60 2024-04-26T20:00:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.626+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66680693c0 con 0x7f66680d5b60 2024-04-26T20:00:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.627+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f665000bc20 con 0x7f66680d5b60 2024-04-26T20:00:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.628+0000 7f665ea00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 0x7f664406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.628+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f665008bf90 con 0x7f66680d5b60 2024-04-26T20:00:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.628+0000 7f6666a00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 0x7f664406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:04.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.629+0000 7f6666a00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 0x7f664406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f665802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.632+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f665005a3b0 con 0x7f66680d5b60 2024-04-26T20:00:04.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.877+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f666806d330 con 0x7f66680d5b60 2024-04-26T20:00:04.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.877+0000 7f665ea00640 1 -- 172.21.15.88:0/632684419 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f6650059cb0 con 0x7f66680d5b60 2024-04-26T20:00:04.878 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:00:04.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 msgr2=0x7f664406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 0x7f664406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f665802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:04.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 msgr2=0x7f666814eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:04.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f665000d800 tx=0x7f665000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 shutdown_connections 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f664406c8f0 0x7f664406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f666815f1e0 0x7f666814f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66680d5b60 0x7f666814eee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 --2- 172.21.15.88:0/632684419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66680d2fc0 0x7f6668161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 >> 172.21.15.88:0/632684419 conn(0x7f666809e2d0 msgr2=0x7f66680cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 shutdown_connections 2024-04-26T20:00:04.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:04.880+0000 7f6667e00640 1 -- 172.21.15.88:0/632684419 wait complete. 2024-04-26T20:00:04.898 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:00:04.898 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:00:04.898 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:00:04.899 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:00:04.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:04.899 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:04.948 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42572sec preferred_lft 42572sec 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-04-26T20:00:04.949 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:04.950 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:00:04.950 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:04.950 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:04.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:00:04.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-04-26T20:00:04.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-04-26T20:00:04.951 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:00:04.951 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:05.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:05.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:05.434 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:00:05.501 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-04-26T20:00:05.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:05.505 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-04-26T20:00:05.565 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-04-26T20:00:05.565 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:00:05.565 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:00:05.565 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:00:05.565 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:00:05.567 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:05.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:06.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:06.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:06.288 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:00:06.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:06.331 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-04-26T20:00:06.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:06.413 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:06.413 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:06.413 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:00:06.413 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:06.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:06.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:06.632 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:00:06.632 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:00:06.633 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:06.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:06.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:06.907 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:06.907 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:06.907 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-04-26T20:00:06.907 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:00:06.907 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:06.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:07.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:07.098 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:00:07.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:07.098 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:07.136 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:00:07.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:07.137 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:07.226 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:00:07.342 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:07.392 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:00:07.393 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:00:07.394 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:07.394 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:07.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:07.395 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:00:07.441 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:00:07.441 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:00:07.489 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:00:07.557 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:07.596 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:00:07.597 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:00:07.598 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:00:07.599 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:00:07.599 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:00:07.599 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:00:07.599 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:07.599 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:07.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:07.600 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:00:07.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2024-04-26T20:00:07.695+0000 7fc11e235480 -1 init, newargv = 0x562951d111e0 newargc=15 2024-04-26T20:00:07.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[41228]: starting ceph client 2024-04-26T20:00:07.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[41228]: starting fuse 2024-04-26T20:00:08.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:00:08.880 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:08.931 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:00:08.931 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:00:08.931 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:08.932 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:08.933 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:08.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:08.934 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:00:08.985 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:00:08.986 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:08.986 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-04-26T20:00:08.987 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:00:08.988 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:08.988 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2024-04-26T20:00:08.988 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-04-26T20:00:09.152 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.41228.asok 2024-04-26T20:00:09.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:09.170 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41228.asok 2024-04-26T20:00:09.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:09.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41228.asok status 2024-04-26T20:00:09.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:00:09.379 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-04-26T20:00:09.388 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "41228", 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4635, 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4635 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 256636056 2024-04-26T20:00:09.389 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 256636056 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.4635 192.168.0.1:0/256636056", 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/256636056", 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 7, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 20, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-04-26T20:00:09.390 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-04-26T20:00:09.391 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:00:09.403 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:00:09.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.645+0000 7fd741600640 1 Processor -- start 2024-04-26T20:00:09.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.645+0000 7fd741600640 1 -- start start 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.646+0000 7fd741600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd73c15cdf0 0x7fd73c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.646+0000 7fd741600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0af9a0 con 0x7fd73c06ce80 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.646+0000 7fd741600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0afc80 con 0x7fd73c0cd650 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.646+0000 7fd741600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0ae2c0 con 0x7fd73c15cdf0 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd73b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd73c06ce80 0x7fd73c0b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54318/0 (socket says 172.21.15.88:54318) 2024-04-26T20:00:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd73b400640 1 -- 172.21.15.88:0/3826493922 learned_addr learned my addr 172.21.15.88:0/3826493922 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3826493922 >> v2:172.21.15.132:3300/0 conn(0x7fd73c15cdf0 0x7fd73c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19805009 0 0) 0x7fd73c0af9a0 con 0x7fd73c06ce80 2024-04-26T20:00:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd720003520 con 0x7fd73c06ce80 2024-04-26T20:00:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2382915351 0 0) 0x7fd720003520 con 0x7fd73c06ce80 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 >> v2:172.21.15.132:3300/0 conn(0x7fd73c15cdf0 msgr2=0x7fd73c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.647+0000 7fd739600640 1 --2- 172.21.15.88:0/3826493922 >> v2:172.21.15.132:3300/0 conn(0x7fd73c15cdf0 0x7fd73c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 >> v1:172.21.15.132:6789/0 conn(0x7fd73c0cd650 legacy=0x7fd73c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0aff60 con 0x7fd73c06ce80 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd734001670 con 0x7fd73c06ce80 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 >> v1:172.21.15.88:6789/0 conn(0x7fd73c06ce80 legacy=0x7fd73c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 0x7fd720003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720004800 0x7fd720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7200051c0 0x7fd72000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd72000c090 con 0x7fd720003a00 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd72000c240 con 0x7fd7200051c0 2024-04-26T20:00:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd72000c440 con 0x7fd720004800 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 0x7fd720003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 -- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720004800 msgr2=0x7fd720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720004800 0x7fd720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 -- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7200051c0 msgr2=0x7fd72000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7200051c0 0x7fd72000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73aa00640 1 -- 172.21.15.88:0/3826493922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd720003520 con 0x7fd720003a00 2024-04-26T20:00:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.648+0000 7fd73b400640 1 -- 172.21.15.88:0/3826493922 reap_dead start 2024-04-26T20:00:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.649+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 0x7fd720003e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009aa0 tx=0x7fd720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae432b1ebea1777 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c036070 con 0x7fd720003a00 2024-04-26T20:00:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c031400 con 0x7fd720003a00 2024-04-26T20:00:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd739600640 1 -- 172.21.15.88:0/3826493922 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c009670 con 0x7fd720003a00 2024-04-26T20:00:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 -- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 msgr2=0x7fd720003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 0x7fd720003e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009aa0 tx=0x7fd720005830 comp rx=0 tx=0).stop 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 -- 172.21.15.88:0/3826493922 shutdown_connections 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 --2- 172.21.15.88:0/3826493922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd720003a00 0x7fd720003e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 -- 172.21.15.88:0/3826493922 >> 172.21.15.88:0/3826493922 conn(0x7fd73c0b2d50 msgr2=0x7fd73c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 -- 172.21.15.88:0/3826493922 shutdown_connections 2024-04-26T20:00:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.650+0000 7fd741600640 1 -- 172.21.15.88:0/3826493922 wait complete. 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.652+0000 7fd741600640 1 Processor -- start 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.652+0000 7fd741600640 1 -- start start 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720003a00 0x7fd73c1f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c15d530 0x7fd73c14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c15d1d0 con 0x7fd73c0d0670 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0b05b0 con 0x7fd73c15d530 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.653+0000 7fd741600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c15d350 con 0x7fd720003a00 2024-04-26T20:00:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42748/0 (socket says 172.21.15.88:42748) 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 -- 172.21.15.88:0/3169828693 learned_addr learned my addr 172.21.15.88:0/3169828693 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720003a00 0x7fd73c1f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73a000640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c15d530 0x7fd73c14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 -- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720003a00 msgr2=0x7fd73c1f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720003a00 0x7fd73c1f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 -- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c15d530 msgr2=0x7fd73c14d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c15d530 0x7fd73c14d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.654+0000 7fd73b400640 1 -- 172.21.15.88:0/3169828693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c15daf0 con 0x7fd73c0d0670 2024-04-26T20:00:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd73b400640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd73c073540 tx=0x7fd73400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734007440 con 0x7fd73c0d0670 2024-04-26T20:00:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd734007600 con 0x7fd73c0d0670 2024-04-26T20:00:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c15ddd0 con 0x7fd73c0d0670 2024-04-26T20:00:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734004a20 con 0x7fd73c0d0670 2024-04-26T20:00:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.655+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c15e390 con 0x7fd73c0d0670 2024-04-26T20:00:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.656+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c069540 con 0x7fd73c0d0670 2024-04-26T20:00:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.657+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd734004bc0 con 0x7fd73c0d0670 2024-04-26T20:00:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.658+0000 7fd732a00640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.658+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd73408b280 con 0x7fd73c0d0670 2024-04-26T20:00:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.658+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.659+0000 7fd73aa00640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 0x7fd71c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd72c033000 tx=0x7fd72c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:09.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.663+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7340192a0 con 0x7fd73c0d0670 2024-04-26T20:00:09.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.904+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd73c0adeb0 con 0x7fd73c0d0670 2024-04-26T20:00:09.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.905+0000 7fd732a00640 1 -- 172.21.15.88:0/3169828693 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fd734059610 con 0x7fd73c0d0670 2024-04-26T20:00:09.906 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.907+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 msgr2=0x7fd71c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.907+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 0x7fd71c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd72c033000 tx=0x7fd72c008030 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.907+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 msgr2=0x7fd73c0d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.907+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd73c073540 tx=0x7fd73400c960 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 shutdown_connections 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd73c0d0670 0x7fd73c0d0b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd73c15d530 0x7fd73c14d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd71c06c8f0 0x7fd71c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 --2- 172.21.15.88:0/3169828693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd720003a00 0x7fd73c1f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 >> 172.21.15.88:0/3169828693 conn(0x7fd73c0b2d50 msgr2=0x7fd73c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 shutdown_connections 2024-04-26T20:00:09.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:09.908+0000 7fd741600640 1 -- 172.21.15.88:0/3169828693 wait complete. 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:00:09.926 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:00:09.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:09.926 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-04-26T20:00:09.976 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42567sec preferred_lft 42567sec 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:33:4d:d7:93:b6 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::9c6e:acff:fec3:49a/64 scope link 2024-04-26T20:00:09.977 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.978 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:00:09.978 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:33:4d:d7:93:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:09.978 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c433:4dff:fed7:93b6/64 scope link 2024-04-26T20:00:09.978 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:09.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:09.979 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:00:10.051 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-04-26T20:00:10.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:10.055 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-04-26T20:00:10.106 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-04-26T20:00:10.106 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:00:10.107 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:00:10.107 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:00:10.107 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:00:10.108 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:10.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:10.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:10.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:10.326 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:00:10.374 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:00:10.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:10.375 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-04-26T20:00:10.422 INFO:teuthology.orchestra.run.smithi088.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T20:00:10.423 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:10.423 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:10.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:00:10.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T20:00:10.423 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:10.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:10.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:10.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:10.602 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:10.678 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:f7:94:72:3e:35 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:00:10.679 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:00:10.679 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.679 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecf7:94ff:fe72:3e35/64 scope link 2024-04-26T20:00:10.679 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:10.683 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:00:10.762 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:f7:94:72:3e:35 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecf7:94ff:fe72:3e35/64 scope link 2024-04-26T20:00:10.763 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:10.767 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T20:00:10.767 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:10.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:11.044 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:11.044 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:00:11.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.1 up 2024-04-26T20:00:11.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:00:11.044 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:00:11.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:11.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:11.245 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-26T20:00:11.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:11.245 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:11.288 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:00:11.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:11.288 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:11.382 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:00:11.465 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:11.521 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:00:11.522 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:00:11.523 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:00:11.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:11.524 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:00:11.576 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:00:11.577 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-26T20:00:11.577 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-26T20:00:11.582 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:00:11.651 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:11.693 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:00:11.693 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:00:11.693 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:00:11.693 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:00:11.694 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:00:11.695 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:11.696 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:00:11.696 INFO:teuthology.orchestra.run.smithi088.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:11.696 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:00:11.696 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:00:11.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:11.697 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:00:11.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:2024-04-26T20:00:11.751+0000 7f360ce17480 -1 init, newargv = 0x55d3838261e0 newargc=15 2024-04-26T20:00:11.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:ceph-fuse[41809]: starting ceph client 2024-04-26T20:00:11.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi088.stderr:ceph-fuse[41809]: starting fuse 2024-04-26T20:00:11.783 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:00:11.783 INFO:teuthology.orchestra.run.smithi088.stdout:43 2024-04-26T20:00:11.783 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-04-26T20:00:11.784 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.2")) 2024-04-26T20:00:11.785 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-04-26T20:00:11.974 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.2.41809.asok 2024-04-26T20:00:11.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:11.990 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.41809.asok 2024-04-26T20:00:11.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:11.991 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.41809.asok status 2024-04-26T20:00:12.217 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:00:12.217 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-04-26T20:00:12.217 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:00:12.217 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:00:12.217 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "2", 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "41809", 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4647, 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 4647 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:12.218 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1045801338 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1045801338 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.4647 192.168.0.2:0/1045801338", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.2:0/1045801338", 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 7, 2024-04-26T20:00:12.219 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 20, 2024-04-26T20:00:12.220 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:00:12.220 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-04-26T20:00:12.220 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-04-26T20:00:12.220 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:00:12.232 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:00:12.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.435+0000 7f86f1400640 1 Processor -- start 2024-04-26T20:00:12.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.435+0000 7f86f1400640 1 -- start start 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86f1400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86ec15dec0 0x7f86ec0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86f1400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86ec15e370 con 0x7f86ec06fa90 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86f1400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86ec06fef0 con 0x7f86ec073720 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86f1400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f86ec0701a0 con 0x7f86ec15dec0 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86f0a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86ec15dec0 0x7f86ec0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86eb400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f86ec06fa90 0x7f86ec06ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54320/0 (socket says 172.21.15.88:54320) 2024-04-26T20:00:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.437+0000 7f86eb400640 1 -- 172.21.15.88:0/2738612240 learned_addr learned my addr 172.21.15.88:0/2738612240 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:12.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.438+0000 7f86eaa00640 1 -- 172.21.15.88:0/2738612240 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 507777045 0 0) 0x7f86d8001110 con 0x7f86ec06fa90 2024-04-26T20:00:12.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.438+0000 7f86f0a00640 1 -- 172.21.15.88:0/2738612240 >> v1:172.21.15.132:6789/0 conn(0x7f86ec073720 legacy=0x7f86ec073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.438+0000 7f86f0a00640 1 -- 172.21.15.88:0/2738612240 >> v1:172.21.15.88:6789/0 conn(0x7f86ec06fa90 legacy=0x7f86ec06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.438+0000 7f86f0a00640 1 -- 172.21.15.88:0/2738612240 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec06df80 con 0x7f86ec15dec0 2024-04-26T20:00:12.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.439+0000 7f86f0a00640 1 --2- 172.21.15.88:0/2738612240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec0cb130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009d90 tx=0x7f86e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841458df975e6221 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:12.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86eaa00640 1 -- 172.21.15.88:0/2738612240 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007740 con 0x7f86ec15dec0 2024-04-26T20:00:12.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86eaa00640 1 -- 172.21.15.88:0/2738612240 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e40078e0 con 0x7f86ec15dec0 2024-04-26T20:00:12.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86eaa00640 1 -- 172.21.15.88:0/2738612240 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007cb0 con 0x7f86ec15dec0 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86f1400640 1 -- 172.21.15.88:0/2738612240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 msgr2=0x7f86ec0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86f1400640 1 --2- 172.21.15.88:0/2738612240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec0cb130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009d90 tx=0x7f86e402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86f1400640 1 -- 172.21.15.88:0/2738612240 shutdown_connections 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.440+0000 7f86f1400640 1 --2- 172.21.15.88:0/2738612240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec0cb130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.441+0000 7f86f1400640 1 -- 172.21.15.88:0/2738612240 >> 172.21.15.88:0/2738612240 conn(0x7f86ec0d0ae0 msgr2=0x7f86ec0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.441+0000 7f86f1400640 1 -- 172.21.15.88:0/2738612240 shutdown_connections 2024-04-26T20:00:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.441+0000 7f86f1400640 1 -- 172.21.15.88:0/2738612240 wait complete. 2024-04-26T20:00:12.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.443+0000 7f86f1400640 1 Processor -- start 2024-04-26T20:00:12.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.443+0000 7f86f1400640 1 -- start start 2024-04-26T20:00:12.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86ec06fa90 0x7f86ec1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:12.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86ec073720 0x7f86ec1efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec146910 con 0x7f86ec15dec0 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec06e540 con 0x7f86ec073720 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.444+0000 7f86f1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec146a90 con 0x7f86ec06fa90 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42770/0 (socket says 172.21.15.88:42770) 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 -- 172.21.15.88:0/821236577 learned_addr learned my addr 172.21.15.88:0/821236577 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86eb400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86ec073720 0x7f86ec1efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86ebe00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86ec06fa90 0x7f86ec1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 -- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86ec06fa90 msgr2=0x7f86ec1f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86ec06fa90 0x7f86ec1f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 -- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86ec073720 msgr2=0x7f86ec1efeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86ec073720 0x7f86ec1efeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.445+0000 7f86f0a00640 1 -- 172.21.15.88:0/821236577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec159530 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86f0a00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009eb0 tx=0x7f86e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4040070 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e4031c90 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ec1f39a0 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4045680 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.446+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ec158770 con 0x7f86ec15dec0 2024-04-26T20:00:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.447+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ec0693c0 con 0x7f86ec15dec0 2024-04-26T20:00:12.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.448+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86e4039040 con 0x7f86ec15dec0 2024-04-26T20:00:12.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.449+0000 7f86e3e00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:12.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.449+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f86e40ad580 con 0x7f86ec15dec0 2024-04-26T20:00:12.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.450+0000 7f86ebe00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:12.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.450+0000 7f86ebe00640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 0x7f86cc06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007fd0 tx=0x7f86dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:12.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.454+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86e407b9d0 con 0x7f86ec15dec0 2024-04-26T20:00:12.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.697+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86ec1f03f0 con 0x7f86ec15dec0 2024-04-26T20:00:12.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.698+0000 7f86e3e00640 1 -- 172.21.15.88:0/821236577 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f86e403ae50 con 0x7f86ec15dec0 2024-04-26T20:00:12.699 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:00:12.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.700+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 msgr2=0x7f86cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.700+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 0x7f86cc06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007fd0 tx=0x7f86dc009040 comp rx=0 tx=0).stop 2024-04-26T20:00:12.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.700+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 msgr2=0x7f86ec159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.700+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009eb0 tx=0x7f86e4004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 shutdown_connections 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86ec15dec0 0x7f86ec159be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86ec073720 0x7f86ec1efeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 --2- 172.21.15.88:0/821236577 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86ec06fa90 0x7f86ec1f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 >> 172.21.15.88:0/821236577 conn(0x7f86ec0d0ae0 msgr2=0x7f86ec0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 shutdown_connections 2024-04-26T20:00:12.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:12.701+0000 7f86f1400640 1 -- 172.21.15.88:0/821236577 wait complete. 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:00:12.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:00:12.720 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:00:12.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:12.720 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-26T20:00:12.777 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:12.777 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:12.777 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:12.777 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:12.778 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-26T20:00:12.779 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:12.780 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:00:12.781 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:12.887 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:13.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:13.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:13.271 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:00:13.336 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T20:00:13.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:13.340 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-26T20:00:13.399 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-26T20:00:13.400 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:00:13.400 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:00:13.400 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:00:13.400 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:00:13.400 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:13.501 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:14.130 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:14.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:14.141 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:00:14.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:14.193 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-04-26T20:00:14.246 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:14.246 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:14.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:14.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T20:00:14.246 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:14.354 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:14.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:14.433 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T20:00:14.433 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:14.433 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:14.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T20:00:14.434 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:00:14.434 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T20:00:14.434 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T20:00:14.434 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T20:00:14.434 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:14.534 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:14.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:14.734 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:14.734 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:14.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-26T20:00:14.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:00:14.734 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:14.826 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:14.926 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:14.935 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:00:14.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:14.935 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:14.979 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:00:14.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:14.979 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:15.072 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:00:15.151 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:15.200 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:00:15.201 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:00:15.202 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:15.203 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:15.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:15.204 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:00:15.252 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:00:15.253 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:00:15.257 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:00:15.322 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:00:15.363 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:00:15.364 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:15.365 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:15.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:15.366 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:00:15.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:2024-04-26T20:00:15.433+0000 7f2b5d121480 -1 init, newargv = 0x562020dd2a50 newargc=15 2024-04-26T20:00:15.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[38854]: starting ceph client 2024-04-26T20:00:15.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[38854]: starting fuse 2024-04-26T20:00:15.452 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:00:15.452 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:00:15.452 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:00:15.452 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-04-26T20:00:15.453 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.1")) 2024-04-26T20:00:15.454 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-04-26T20:00:15.634 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.1.38854.asok 2024-04-26T20:00:15.638 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:15.650 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38854.asok 2024-04-26T20:00:15.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:15.651 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38854.asok status 2024-04-26T20:00:15.878 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "1", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "38854", 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 4412, 2024-04-26T20:00:15.879 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 4412 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3007669866 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:00:15.880 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3007669866 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.4412 192.168.0.1:0/3007669866", 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.1:0/3007669866", 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 7, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 20, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-04-26T20:00:15.881 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-26T20:00:15.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:00:16.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.103+0000 7f050e200640 1 Processor -- start 2024-04-26T20:00:16.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.103+0000 7f050e200640 1 -- start start 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.104+0000 7f050e200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f050806fa90 0x7f050815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.104+0000 7f050e200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050806fef0 con 0x7f0508073720 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.104+0000 7f050e200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05080701a0 con 0x7f050806ed40 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.104+0000 7f050e200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f050806df80 con 0x7f050806fa90 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f050ce00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0508073720 0x7f0508073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:54334/0 (socket says 172.21.15.88:54334) 2024-04-26T20:00:16.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f050ce00640 1 -- 172.21.15.88:0/3688878669 learned_addr learned my addr 172.21.15.88:0/3688878669 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:16.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923034013 0 0) 0x7f050806fef0 con 0x7f0508073720 2024-04-26T20:00:16.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04ec003520 con 0x7f0508073720 2024-04-26T20:00:16.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905761614 0 0) 0x7f04ec003520 con 0x7f0508073720 2024-04-26T20:00:16.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 >> v2:172.21.15.132:3300/0 conn(0x7f050806fa90 msgr2=0x7f050815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.105+0000 7f0507e00640 1 --2- 172.21.15.88:0/3688878669 >> v2:172.21.15.132:3300/0 conn(0x7f050806fa90 0x7f050815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 --2- 172.21.15.88:0/3688878669 >> v2:172.21.15.132:3300/0 conn(0x7f050806fa90 0x7f050815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 >> v1:172.21.15.132:6789/0 conn(0x7f050806ed40 legacy=0x7f05080cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050806e260 con 0x7f0508073720 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f04f80019f0 con 0x7f0508073720 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 >> v1:172.21.15.88:6789/0 conn(0x7f0508073720 legacy=0x7f0508073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 0x7f04ec003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec004800 0x7f04ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0052e0 0x7f04ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f050806df80 con 0x7f04ec0039b0 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec00c1e0 con 0x7f04ec004800 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f04ec00c3e0 con 0x7f04ec0052e0 2024-04-26T20:00:16.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f0507e00640 1 --2- 172.21.15.88:0/3688878669 >> v2:172.21.15.132:3300/0 conn(0x7f050806fa90 0x7f050815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f050ce00640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 0x7f04ec003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f050ce00640 1 -- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0052e0 msgr2=0x7f04ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.106+0000 7f050d800640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0052e0 0x7f04ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.107+0000 7f050ce00640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0052e0 0x7f04ec00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.107+0000 7f050ce00640 1 -- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec004800 msgr2=0x7f04ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.107+0000 7f050ce00640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ec004800 0x7f04ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.107+0000 7f050ce00640 1 -- 172.21.15.88:0/3688878669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec003520 con 0x7f04ec0039b0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.107+0000 7f050d800640 1 -- 172.21.15.88:0/3688878669 reap_dead start 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050ce00640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 0x7f04ec003db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a850 tx=0x7f04ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e098365ed0a0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800a4a0 con 0x7f04ec0039b0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f802f070 con 0x7f04ec0039b0 2024-04-26T20:00:16.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f0507400640 1 -- 172.21.15.88:0/3688878669 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f802bcf0 con 0x7f04ec0039b0 2024-04-26T20:00:16.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050e200640 1 -- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 msgr2=0x7f04ec003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050e200640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 0x7f04ec003db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a850 tx=0x7f04ec005830 comp rx=0 tx=0).stop 2024-04-26T20:00:16.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050e200640 1 -- 172.21.15.88:0/3688878669 shutdown_connections 2024-04-26T20:00:16.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050e200640 1 --2- 172.21.15.88:0/3688878669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ec0039b0 0x7f04ec003db0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.108+0000 7f050e200640 1 -- 172.21.15.88:0/3688878669 >> 172.21.15.88:0/3688878669 conn(0x7f05080d0ae0 msgr2=0x7f05080d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:16.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.109+0000 7f050e200640 1 -- 172.21.15.88:0/3688878669 shutdown_connections 2024-04-26T20:00:16.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.109+0000 7f050e200640 1 -- 172.21.15.88:0/3688878669 wait complete. 2024-04-26T20:00:16.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.110+0000 7f050e200640 1 Processor -- start 2024-04-26T20:00:16.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.110+0000 7f050e200640 1 -- start start 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0039b0 0x7f05080b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0508141680 0x7f05081515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0508146910 con 0x7f0508151b00 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f050806f1f0 con 0x7f0508141680 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.111+0000 7f050e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0508146a90 con 0x7f04ec0039b0 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42790/0 (socket says 172.21.15.88:42790) 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 -- 172.21.15.88:0/393607451 learned_addr learned my addr 172.21.15.88:0/393607451 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050ce00640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0039b0 0x7f05080b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f0507e00640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0508141680 0x7f05081515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 -- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0039b0 msgr2=0x7f05080b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0039b0 0x7f05080b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 -- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0508141680 msgr2=0x7f05081515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0508141680 0x7f05081515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 -- 172.21.15.88:0/393607451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050815c8d0 con 0x7f0508151b00 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.112+0000 7f050d800640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05080b6c20 tx=0x7f050000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.113+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500019070 con 0x7f0508151b00 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.113+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500005250 con 0x7f0508151b00 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.113+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500012600 con 0x7f0508151b00 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.113+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0508141b00 con 0x7f0508151b00 2024-04-26T20:00:16.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.113+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05080b5330 con 0x7f0508151b00 2024-04-26T20:00:16.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.114+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05080693c0 con 0x7f0508151b00 2024-04-26T20:00:16.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.115+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f050001c070 con 0x7f0508151b00 2024-04-26T20:00:16.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.116+0000 7f0504c00640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:16.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.116+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f050008aa40 con 0x7f0508151b00 2024-04-26T20:00:16.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.116+0000 7f050ce00640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:16.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.117+0000 7f050ce00640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 0x7f04e806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04f8033000 tx=0x7f04f8038040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:16.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.121+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0500058f00 con 0x7f0508151b00 2024-04-26T20:00:16.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.369+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f05080ce270 con 0x7f0508151b00 2024-04-26T20:00:16.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.369+0000 7f0504c00640 1 -- 172.21.15.88:0/393607451 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f050008ae20 con 0x7f0508151b00 2024-04-26T20:00:16.370 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 msgr2=0x7f04e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 0x7f04e806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04f8033000 tx=0x7f04f8038040 comp rx=0 tx=0).stop 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 msgr2=0x7f050814f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f05080b6c20 tx=0x7f050000d890 comp rx=0 tx=0).stop 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 shutdown_connections 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0508151b00 0x7f050814f530 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0508141680 0x7f05081515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f04e806c920 0x7f04e806ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 --2- 172.21.15.88:0/393607451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ec0039b0 0x7f05080b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:16.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 >> 172.21.15.88:0/393607451 conn(0x7f05080d0ae0 msgr2=0x7f050815f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:16.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.372+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 shutdown_connections 2024-04-26T20:00:16.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:16.373+0000 7f050e200640 1 -- 172.21.15.88:0/393607451 wait complete. 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:00:16.391 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:00:16.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:16.392 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-26T20:00:16.450 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:16.450 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:16.450 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:16.451 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ccda:b9ff:fed8:349c/64 scope link 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::8029:7dff:fe56:31fa/64 scope link tentative 2024-04-26T20:00:16.452 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:16.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:16.453 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:00:16.532 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T20:00:16.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:16.536 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-26T20:00:16.586 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-26T20:00:16.586 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:00:16.586 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:00:16.586 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:00:16.586 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:00:16.587 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:16.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:16.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:16.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:16.798 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:00:16.843 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:00:16.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:16.844 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-04-26T20:00:16.938 INFO:teuthology.orchestra.run.smithi132.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T20:00:16.939 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:16.939 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:16.939 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:00:16.939 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T20:00:16.939 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:17.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:17.114 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:17.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:17.123 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:00:17.193 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:17.193 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:17.193 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 62:62:82:42:c8:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::6062:82ff:fe42:c843/64 scope link 2024-04-26T20:00:17.194 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:17.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:00:17.281 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 62:62:82:42:c8:43 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::6062:82ff:fe42:c843/64 scope link 2024-04-26T20:00:17.282 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:00:17.287 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T20:00:17.287 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:17.392 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:17.555 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:17.564 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:00:17.564 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:00:17.564 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.1 up 2024-04-26T20:00:17.564 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:00:17.564 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:00:17.657 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:00:17.755 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:17.764 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-26T20:00:17.764 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:17.764 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:17.810 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:00:17.810 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:00:17.810 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:17.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:00:17.994 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:00:18.047 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:18.048 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:00:18.049 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:00:18.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.050 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:00:18.096 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:00:18.096 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-26T20:00:18.096 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-26T20:00:18.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:00:18.214 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:00:18.258 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:00:18.259 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:00:18.260 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:00:18.261 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:00:18.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.262 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:00:18.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi132.stderr:2024-04-26T20:00:18.323+0000 7f6dba070480 -1 init, newargv = 0x55ca43ad0310 newargc=15 2024-04-26T20:00:18.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi132.stderr:ceph-fuse[39310]: starting ceph client 2024-04-26T20:00:18.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi132.stderr:ceph-fuse[39310]: starting fuse 2024-04-26T20:00:18.347 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:00:18.347 INFO:teuthology.orchestra.run.smithi132.stdout:43 2024-04-26T20:00:18.347 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-04-26T20:00:18.348 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:00:18.349 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.3")) 2024-04-26T20:00:18.350 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-04-26T20:00:18.532 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.3.39310.asok 2024-04-26T20:00:18.537 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:00:18.549 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39310.asok 2024-04-26T20:00:18.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39310.asok status 2024-04-26T20:00:18.775 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-26T20:00:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-04-26T20:00:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:00:18.775 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "3", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "39310", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 4504, 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 4504 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:00:18.776 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 2541054268 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 2541054268 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.4504 192.168.0.2:0/2541054268", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.2:0/2541054268", 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 7, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 20, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:00:18.777 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-04-26T20:00:18.778 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-04-26T20:00:18.778 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-26T20:00:18.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.789 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-04-26T20:00:18.789 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:18.848 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-04-26T20:00:18.848 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:18.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.848 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:18.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.935 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-04-26T20:00:18.935 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:18.989 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-04-26T20:00:18.989 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:18.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:19.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:19.081 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-04-26T20:00:19.081 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:19.139 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-04-26T20:00:19.139 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:19.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:19.139 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:19.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:19.226 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-04-26T20:00:19.226 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:19.277 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-04-26T20:00:19.277 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:19.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:19.278 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:19.353 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T20:00:19.369 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:19.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.576+0000 7f57d2200640 1 Processor -- start 2024-04-26T20:00:19.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.577+0000 7f57d2200640 1 -- start start 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 0x7f57cc0d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 0x7f57cc162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57cc163480 0x7f57cc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc163910 con 0x7f57cc163480 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc0d3f50 con 0x7f57cc15f1b0 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0d36b0 con 0x7f57cc162b00 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d1800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57cc163480 0x7f57cc0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d1800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f57cc163480 0x7f57cc0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48120/0 (socket says 172.21.15.88:48120) 2024-04-26T20:00:19.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.578+0000 7f57d1800640 1 -- 172.21.15.88:0/1574207696 learned_addr learned my addr 172.21.15.88:0/1574207696 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d1800640 1 -- 172.21.15.88:0/1574207696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 msgr2=0x7f57cc162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57cbe00640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 0x7f57cc162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d0e00640 1 --2- 172.21.15.88:0/1574207696 >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 0x7f57cc0d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d1800640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 0x7f57cc162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d1800640 1 -- 172.21.15.88:0/1574207696 >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 msgr2=0x7f57cc0d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d1800640 1 --2- 172.21.15.88:0/1574207696 >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 0x7f57cc0d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d1800640 1 -- 172.21.15.88:0/1574207696 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0d3990 con 0x7f57cc163480 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57cbe00640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 0x7f57cc162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:19.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.579+0000 7f57d0e00640 1 --2- 172.21.15.88:0/1574207696 >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 0x7f57cc0d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:19.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.580+0000 7f57d1800640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc163480 0x7f57cc0cfad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57cc06f570 tx=0x7f57c402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de45b1a1ea25c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:19.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.580+0000 7f57cb400640 1 -- 172.21.15.88:0/1574207696 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4036070 con 0x7f57cc163480 2024-04-26T20:00:19.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.580+0000 7f57cb400640 1 -- 172.21.15.88:0/1574207696 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4031600 con 0x7f57cc163480 2024-04-26T20:00:19.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.580+0000 7f57cb400640 1 -- 172.21.15.88:0/1574207696 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40319d0 con 0x7f57cc163480 2024-04-26T20:00:19.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 -- 172.21.15.88:0/1574207696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc163480 msgr2=0x7f57cc0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc163480 0x7f57cc0cfad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57cc06f570 tx=0x7f57c402a690 comp rx=0 tx=0).stop 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 -- 172.21.15.88:0/1574207696 shutdown_connections 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc163480 0x7f57cc0cfad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 --2- 172.21.15.88:0/1574207696 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc162b00 0x7f57cc162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 --2- 172.21.15.88:0/1574207696 >> v2:172.21.15.132:3300/0 conn(0x7f57cc15f1b0 0x7f57cc0d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 -- 172.21.15.88:0/1574207696 >> 172.21.15.88:0/1574207696 conn(0x7f57cc09e230 msgr2=0x7f57cc0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 -- 172.21.15.88:0/1574207696 shutdown_connections 2024-04-26T20:00:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.581+0000 7f57d2200640 1 -- 172.21.15.88:0/1574207696 wait complete. 2024-04-26T20:00:19.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.583+0000 7f57d2200640 1 Processor -- start 2024-04-26T20:00:19.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.583+0000 7f57d2200640 1 -- start start 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57cc162b00 0x7f57cc0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc163480 0x7f57cc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0d33a0 con 0x7f57cc15f1b0 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0b7d90 con 0x7f57cc162b00 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d2200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0d0010 con 0x7f57cc163480 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48122/0 (socket says 172.21.15.88:48122) 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.584+0000 7f57d0e00640 1 -- 172.21.15.88:0/3826660606 learned_addr learned my addr 172.21.15.88:0/3826660606 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d1800640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc163480 0x7f57cc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57cbe00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57cc162b00 0x7f57cc0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 -- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc163480 msgr2=0x7f57cc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc163480 0x7f57cc0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 -- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57cc162b00 msgr2=0x7f57cc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57cc162b00 0x7f57cc0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 -- 172.21.15.88:0/3826660606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0cbee0 con 0x7f57cc15f1b0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d0e00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c7c0 tx=0x7f57c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0015070 con 0x7f57cc15f1b0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0007a30 con 0x7f57cc15f1b0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc0cc180 con 0x7f57cc15f1b0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.585+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0007e00 con 0x7f57cc15f1b0 2024-04-26T20:00:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.586+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc161ec0 con 0x7f57cc15f1b0 2024-04-26T20:00:19.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.587+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc0693c0 con 0x7f57cc15f1b0 2024-04-26T20:00:19.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.588+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57c0018070 con 0x7f57cc15f1b0 2024-04-26T20:00:19.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.589+0000 7f57c8c00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:19.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.589+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f57c008ba50 con 0x7f57cc15f1b0 2024-04-26T20:00:19.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.589+0000 7f57cbe00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:19.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.590+0000 7f57cbe00640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 0x7f57ac06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc15a480 tx=0x7f57b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:19.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.593+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57c0059e70 con 0x7f57cc15f1b0 2024-04-26T20:00:19.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.794+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57cc08d090 con 0x7f57cc15f1b0 2024-04-26T20:00:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.795+0000 7f57c8c00640 1 -- 172.21.15.88:0/3826660606 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f57c0059770 con 0x7f57cc15f1b0 2024-04-26T20:00:19.796 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:19.797 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:19.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 msgr2=0x7f57ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:19.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 0x7f57ac06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57cc15a480 tx=0x7f57b8008040 comp rx=0 tx=0).stop 2024-04-26T20:00:19.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 msgr2=0x7f57cc156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:19.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c7c0 tx=0x7f57c000cc80 comp rx=0 tx=0).stop 2024-04-26T20:00:19.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 shutdown_connections 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57ac06c8f0 0x7f57ac06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f57cc163480 0x7f57cc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f57cc162b00 0x7f57cc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 --2- 172.21.15.88:0/3826660606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57cc15f1b0 0x7f57cc156930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.798+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 >> 172.21.15.88:0/3826660606 conn(0x7f57cc09e230 msgr2=0x7f57cc15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.799+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 shutdown_connections 2024-04-26T20:00:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:19.799+0000 7f57d2200640 1 -- 172.21.15.88:0/3826660606 wait complete. 2024-04-26T20:00:19.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:19.820 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:20.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.028+0000 7f39a5400640 1 Processor -- start 2024-04-26T20:00:20.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.028+0000 7f39a5400640 1 -- start start 2024-04-26T20:00:20.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.029+0000 7f39a5400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 0x7f39a006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f39a5400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f39a00add80 0x7f39a006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f39a5400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a00af800 con 0x7f39a00b5ca0 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f39a5400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00ae340 con 0x7f39a00add80 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f39a5400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00afaa0 con 0x7f39a015d100 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f399de00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f39a00b5ca0 0x7f39a0072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55670/0 (socket says 172.21.15.88:55670) 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f399de00640 1 -- 172.21.15.88:0/2324045837 learned_addr learned my addr 172.21.15.88:0/2324045837 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f399f200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f39a00add80 0x7f39a006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.030+0000 7f399e800640 1 --2- 172.21.15.88:0/2324045837 >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 0x7f39a006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399f200640 1 -- 172.21.15.88:0/2324045837 >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 msgr2=0x7f39a006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399f200640 1 --2- 172.21.15.88:0/2324045837 >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 0x7f39a006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399f200640 1 -- 172.21.15.88:0/2324045837 >> v1:172.21.15.88:6789/0 conn(0x7f39a00b5ca0 legacy=0x7f39a0072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399f200640 1 -- 172.21.15.88:0/2324045837 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a00afd80 con 0x7f39a00add80 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399e800640 1 --2- 172.21.15.88:0/2324045837 >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 0x7f39a006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:20.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.031+0000 7f399d400640 1 -- 172.21.15.88:0/2324045837 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835944975 0 0) 0x7f39a00af800 con 0x7f39a00b5ca0 2024-04-26T20:00:20.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.032+0000 7f399f200640 1 --2- 172.21.15.88:0/2324045837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a00add80 0x7f39a006ccb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39a00730d0 tx=0x7f399802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e17fe517a15a5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.032+0000 7f399d400640 1 -- 172.21.15.88:0/2324045837 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998037070 con 0x7f39a00add80 2024-04-26T20:00:20.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.032+0000 7f399d400640 1 -- 172.21.15.88:0/2324045837 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998032440 con 0x7f39a00add80 2024-04-26T20:00:20.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.032+0000 7f399d400640 1 -- 172.21.15.88:0/2324045837 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998032810 con 0x7f39a00add80 2024-04-26T20:00:20.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.032+0000 7f39a5400640 1 -- 172.21.15.88:0/2324045837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a00add80 msgr2=0x7f39a006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 --2- 172.21.15.88:0/2324045837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a00add80 0x7f39a006ccb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39a00730d0 tx=0x7f399802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 -- 172.21.15.88:0/2324045837 shutdown_connections 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 --2- 172.21.15.88:0/2324045837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a00add80 0x7f39a006ccb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 --2- 172.21.15.88:0/2324045837 >> v2:172.21.15.132:3300/0 conn(0x7f39a015d100 0x7f39a006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 -- 172.21.15.88:0/2324045837 >> 172.21.15.88:0/2324045837 conn(0x7f39a00b2d50 msgr2=0x7f39a00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 -- 172.21.15.88:0/2324045837 shutdown_connections 2024-04-26T20:00:20.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.033+0000 7f39a5400640 1 -- 172.21.15.88:0/2324045837 wait complete. 2024-04-26T20:00:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.034+0000 7f39a5400640 1 Processor -- start 2024-04-26T20:00:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 -- start start 2024-04-26T20:00:20.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a00add80 0x7f39a014e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a00b5ca0 0x7f39a006f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39a009ee50 con 0x7f39a015d100 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00b0340 con 0x7f39a00b5ca0 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.035+0000 7f39a5400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f39a0160b70 con 0x7f39a00add80 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48128/0 (socket says 172.21.15.88:48128) 2024-04-26T20:00:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 -- 172.21.15.88:0/2590367737 learned_addr learned my addr 172.21.15.88:0/2590367737 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399de00640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a00b5ca0 0x7f39a006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399e800640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a00add80 0x7f39a014e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 -- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a00add80 msgr2=0x7f39a014e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a00add80 0x7f39a014e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 -- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a00b5ca0 msgr2=0x7f39a006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a00b5ca0 0x7f39a006f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.036+0000 7f399f200640 1 -- 172.21.15.88:0/2590367737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a006e010 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f399f200640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3998035040 tx=0x7f39980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998037050 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998038040 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998006e10 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a006e2f0 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.037+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a006e8b0 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.038+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a015de30 con 0x7f39a015d100 2024-04-26T20:00:20.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.039+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3998030980 con 0x7f39a015d100 2024-04-26T20:00:20.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.040+0000 7f3996a00640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 0x7f397c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.040+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f39980ac680 con 0x7f39a015d100 2024-04-26T20:00:20.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.041+0000 7f399e800640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 0x7f397c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.041+0000 7f399e800640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 0x7f397c06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3988006fd0 tx=0x7f3988008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.045+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39a015de30 con 0x7f39a015d100 2024-04-26T20:00:20.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.242+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39a0073d00 con 0x7f39a015d100 2024-04-26T20:00:20.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.243+0000 7f3996a00640 1 -- 172.21.15.88:0/2590367737 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3998032e20 con 0x7f39a015d100 2024-04-26T20:00:20.244 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 msgr2=0x7f397c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 0x7f397c06edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3988006fd0 tx=0x7f3988008040 comp rx=0 tx=0).stop 2024-04-26T20:00:20.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 msgr2=0x7f39a0153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3998035040 tx=0x7f39980046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 shutdown_connections 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a015d100 0x7f39a0153530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a00b5ca0 0x7f39a006f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f397c06c8f0 0x7f397c06edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.245+0000 7f39a5400640 1 --2- 172.21.15.88:0/2590367737 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39a00add80 0x7f39a014e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.246+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 >> 172.21.15.88:0/2590367737 conn(0x7f39a00b2d50 msgr2=0x7f39a00b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.246+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 shutdown_connections 2024-04-26T20:00:20.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.246+0000 7f39a5400640 1 -- 172.21.15.88:0/2590367737 wait complete. 2024-04-26T20:00:20.265 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:20.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.465+0000 7f509f200640 1 Processor -- start 2024-04-26T20:00:20.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.465+0000 7f509f200640 1 -- start start 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509f200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f50a015c450 0x7f50a00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f50a015ee60 0x7f50a0165a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509f200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a015d970 con 0x7f50a006dfe0 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509f200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f50a006e470 con 0x7f50a015c450 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f50a00b4fe0 con 0x7f50a015ee60 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509de00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f50a015c450 0x7f50a00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509d400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f50a006dfe0 0x7f50a015c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55682/0 (socket says 172.21.15.88:55682) 2024-04-26T20:00:20.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.467+0000 7f509d400640 1 -- 172.21.15.88:0/2869787161 learned_addr learned my addr 172.21.15.88:0/2869787161 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:20.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509e800640 1 --2- 172.21.15.88:0/2869787161 >> v2:172.21.15.132:3300/0 conn(0x7f50a015ee60 0x7f50a0165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509de00640 1 -- 172.21.15.88:0/2869787161 >> v2:172.21.15.132:3300/0 conn(0x7f50a015ee60 msgr2=0x7f50a0165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509de00640 1 --2- 172.21.15.88:0/2869787161 >> v2:172.21.15.132:3300/0 conn(0x7f50a015ee60 0x7f50a0165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509de00640 1 -- 172.21.15.88:0/2869787161 >> v1:172.21.15.88:6789/0 conn(0x7f50a006dfe0 legacy=0x7f50a015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509de00640 1 -- 172.21.15.88:0/2869787161 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a00b52c0 con 0x7f50a015c450 2024-04-26T20:00:20.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.468+0000 7f509ca00640 1 -- 172.21.15.88:0/2869787161 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824918128 0 0) 0x7f50a015d970 con 0x7f50a006dfe0 2024-04-26T20:00:20.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.469+0000 7f509de00640 1 --2- 172.21.15.88:0/2869787161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a015c450 0x7f50a00b4920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50a00741f0 tx=0x7f509002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c66b98286ae018 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.469+0000 7f509ca00640 1 -- 172.21.15.88:0/2869787161 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090007740 con 0x7f50a015c450 2024-04-26T20:00:20.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.469+0000 7f509ca00640 1 -- 172.21.15.88:0/2869787161 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50900078e0 con 0x7f50a015c450 2024-04-26T20:00:20.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.469+0000 7f509ca00640 1 -- 172.21.15.88:0/2869787161 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090007cb0 con 0x7f50a015c450 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 -- 172.21.15.88:0/2869787161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a015c450 msgr2=0x7f50a00b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 --2- 172.21.15.88:0/2869787161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a015c450 0x7f50a00b4920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50a00741f0 tx=0x7f509002a680 comp rx=0 tx=0).stop 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 -- 172.21.15.88:0/2869787161 shutdown_connections 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 --2- 172.21.15.88:0/2869787161 >> v2:172.21.15.132:3300/0 conn(0x7f50a015ee60 0x7f50a0165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 --2- 172.21.15.88:0/2869787161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a015c450 0x7f50a00b4920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 -- 172.21.15.88:0/2869787161 >> 172.21.15.88:0/2869787161 conn(0x7f50a00b2900 msgr2=0x7f50a00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 -- 172.21.15.88:0/2869787161 shutdown_connections 2024-04-26T20:00:20.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.470+0000 7f509f200640 1 -- 172.21.15.88:0/2869787161 wait complete. 2024-04-26T20:00:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.472+0000 7f509f200640 1 Processor -- start 2024-04-26T20:00:20.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.472+0000 7f509f200640 1 -- start start 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a015c450 0x7f50a0141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50a015ee60 0x7f50a014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50a00b5a00 con 0x7f50a006dfe0 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f50a00b5880 con 0x7f50a015c450 2024-04-26T20:00:20.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.473+0000 7f509f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f50a015cf40 con 0x7f50a015ee60 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48152/0 (socket says 172.21.15.88:48152) 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 -- 172.21.15.88:0/229600688 learned_addr learned my addr 172.21.15.88:0/229600688 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 -- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50a015ee60 msgr2=0x7f50a014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50a015ee60 0x7f50a014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 -- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a015c450 msgr2=0x7f50a0141860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509d400640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a015c450 0x7f50a0141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a015c450 0x7f50a0141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 -- 172.21.15.88:0/229600688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a01677d0 con 0x7f50a006dfe0 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.474+0000 7f509de00640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f509002ac70 tx=0x7f5090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.475+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090040070 con 0x7f50a006dfe0 2024-04-26T20:00:20.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.475+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5090031c40 con 0x7f50a006dfe0 2024-04-26T20:00:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.475+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509003c420 con 0x7f50a006dfe0 2024-04-26T20:00:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.475+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a01674f0 con 0x7f50a006dfe0 2024-04-26T20:00:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.475+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a0142570 con 0x7f50a006dfe0 2024-04-26T20:00:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.476+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a00cd720 con 0x7f50a006dfe0 2024-04-26T20:00:20.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.477+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5090039040 con 0x7f50a006dfe0 2024-04-26T20:00:20.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.478+0000 7f5096000640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.478+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f50900ad1c0 con 0x7f50a006dfe0 2024-04-26T20:00:20.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.478+0000 7f509d400640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.479+0000 7f509d400640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 0x7f507c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50a01597b0 tx=0x7f5088009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.482+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f50a00cd720 con 0x7f50a006dfe0 2024-04-26T20:00:20.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.679+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50a00693c0 con 0x7f50a006dfe0 2024-04-26T20:00:20.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.680+0000 7f5096000640 1 -- 172.21.15.88:0/229600688 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f50a01677d0 con 0x7f50a006dfe0 2024-04-26T20:00:20.681 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 msgr2=0x7f507c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 0x7f507c06edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f50a01597b0 tx=0x7f5088009040 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 msgr2=0x7f50a0166f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f509002ac70 tx=0x7f5090004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 shutdown_connections 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f507c06c8f0 0x7f507c06edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f50a015ee60 0x7f50a014ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a015c450 0x7f50a0141860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 --2- 172.21.15.88:0/229600688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a006dfe0 0x7f50a0166f10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 >> 172.21.15.88:0/229600688 conn(0x7f50a00b2900 msgr2=0x7f50a015a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 shutdown_connections 2024-04-26T20:00:20.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.683+0000 7f509f200640 1 -- 172.21.15.88:0/229600688 wait complete. 2024-04-26T20:00:20.702 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:20.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.897+0000 7f85efe00640 1 Processor -- start 2024-04-26T20:00:20.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.897+0000 7f85efe00640 1 -- start start 2024-04-26T20:00:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.898+0000 7f85efe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f85f01467a0 0x7f85f015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.898+0000 7f85efe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85f00b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.898+0000 7f85efe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85f015e3e0 con 0x7f85f00b5ca0 2024-04-26T20:00:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.898+0000 7f85efe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f85f0146b80 con 0x7f85f01467a0 2024-04-26T20:00:20.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.898+0000 7f85efe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f85f0161730 con 0x7f85f00d1470 2024-04-26T20:00:20.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85e7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f85f00b5ca0 0x7f85f00d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55692/0 (socket says 172.21.15.88:55692) 2024-04-26T20:00:20.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85e7e00640 1 -- 172.21.15.88:0/1435461156 learned_addr learned my addr 172.21.15.88:0/1435461156 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85eea00640 1 --2- 172.21.15.88:0/1435461156 >> v2:172.21.15.132:3300/0 conn(0x7f85f01467a0 0x7f85f015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85ef400640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85f00b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009418168 0 0) 0x7f85d8001110 con 0x7f85f00b5ca0 2024-04-26T20:00:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.899+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85d4003520 con 0x7f85f00b5ca0 2024-04-26T20:00:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1287219722 0 0) 0x7f85d4003520 con 0x7f85f00b5ca0 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 msgr2=0x7f85f00b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85f00b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 >> v2:172.21.15.132:3300/0 conn(0x7f85f01467a0 msgr2=0x7f85f015dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 --2- 172.21.15.88:0/1435461156 >> v2:172.21.15.132:3300/0 conn(0x7f85f01467a0 0x7f85f015dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f0160b20 con 0x7f85f00b5ca0 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f85d8001840 con 0x7f85f00b5ca0 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 >> v1:172.21.15.88:6789/0 conn(0x7f85f00b5ca0 legacy=0x7f85f00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d4004d40 0x7f85d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0146b80 con 0x7f85d40039b0 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f85f0161730 con 0x7f85d4004d40 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f85d400ba70 con 0x7f85f00d1470 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85eea00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85eea00640 1 -- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 msgr2=0x7f85d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.900+0000 7f85e7e00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.901+0000 7f85eea00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f00d1470 0x7f85d4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.901+0000 7f85eea00640 1 -- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d4004d40 msgr2=0x7f85d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.901+0000 7f85ef400640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d4004d40 0x7f85d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.901+0000 7f85eea00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85d4004d40 0x7f85d40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.901+0000 7f85eea00640 1 -- 172.21.15.88:0/1435461156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d4003520 con 0x7f85d40039b0 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85eea00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85d4003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009ac0 tx=0x7f85d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c6cf0ac96df349 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0036070 con 0x7f85d40039b0 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e0031400 con 0x7f85d40039b0 2024-04-26T20:00:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85ee000640 1 -- 172.21.15.88:0/1435461156 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0009690 con 0x7f85d40039b0 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85efe00640 1 -- 172.21.15.88:0/1435461156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 msgr2=0x7f85d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85efe00640 1 --2- 172.21.15.88:0/1435461156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85d4003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009ac0 tx=0x7f85d4005830 comp rx=0 tx=0).stop 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85ef400640 1 -- 172.21.15.88:0/1435461156 reap_dead start 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85efe00640 1 -- 172.21.15.88:0/1435461156 shutdown_connections 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.902+0000 7f85efe00640 1 -- 172.21.15.88:0/1435461156 >> 172.21.15.88:0/1435461156 conn(0x7f85f006c530 msgr2=0x7f85f006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.903+0000 7f85efe00640 1 -- 172.21.15.88:0/1435461156 shutdown_connections 2024-04-26T20:00:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.903+0000 7f85efe00640 1 -- 172.21.15.88:0/1435461156 wait complete. 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.904+0000 7f85efe00640 1 Processor -- start 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.904+0000 7f85efe00640 1 -- start start 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f0072e90 0x7f85f0073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f01310a0 0x7f85f0131520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01613c0 con 0x7f85d40039b0 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f85f01610e0 con 0x7f85f0072e90 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.905+0000 7f85efe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f85f00cf100 con 0x7f85f01310a0 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48170/0 (socket says 172.21.15.88:48170) 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 -- 172.21.15.88:0/3172664334 learned_addr learned my addr 172.21.15.88:0/3172664334 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 -- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f01310a0 msgr2=0x7f85f0131520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f01310a0 0x7f85f0131520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 -- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f0072e90 msgr2=0x7f85f0073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85e7e00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f0072e90 0x7f85f0073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f0072e90 0x7f85f0073310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 -- 172.21.15.88:0/3172664334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f0143800 con 0x7f85d40039b0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.906+0000 7f85eea00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85e0034040 tx=0x7f85e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.907+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0040020 con 0x7f85d40039b0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.907+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e0003950 con 0x7f85d40039b0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.907+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f0142a40 con 0x7f85d40039b0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.907+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0036040 con 0x7f85d40039b0 2024-04-26T20:00:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.907+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f0143000 con 0x7f85d40039b0 2024-04-26T20:00:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.908+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f00693c0 con 0x7f85d40039b0 2024-04-26T20:00:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.909+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85e0043020 con 0x7f85d40039b0 2024-04-26T20:00:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.910+0000 7f85e6a00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 0x7f85cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.910+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85e00abc20 con 0x7f85d40039b0 2024-04-26T20:00:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.910+0000 7f85e7e00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 0x7f85cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.910+0000 7f85e7e00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 0x7f85cc06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85d80012f0 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:20.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:20.914+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85e004a070 con 0x7f85d40039b0 2024-04-26T20:00:21.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.102+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85f006f7a0 con 0x7f85d40039b0 2024-04-26T20:00:21.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.103+0000 7f85e6a00640 1 -- 172.21.15.88:0/3172664334 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f85e007a050 con 0x7f85d40039b0 2024-04-26T20:00:21.103 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 msgr2=0x7f85cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 0x7f85cc06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85d80012f0 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2024-04-26T20:00:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 msgr2=0x7f85f01f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85e0034040 tx=0x7f85e0004730 comp rx=0 tx=0).stop 2024-04-26T20:00:21.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 shutdown_connections 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f85cc06c920 0x7f85cc06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f85f01310a0 0x7f85f0131520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85f0072e90 0x7f85f0073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 --2- 172.21.15.88:0/3172664334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85d40039b0 0x7f85f01f4ad0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.105+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 >> 172.21.15.88:0/3172664334 conn(0x7f85f006c530 msgr2=0x7f85f00b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.106+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 shutdown_connections 2024-04-26T20:00:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.106+0000 7f85efe00640 1 -- 172.21.15.88:0/3172664334 wait complete. 2024-04-26T20:00:21.124 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:21.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.314+0000 7ff1c4c00640 1 Processor -- start 2024-04-26T20:00:21.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.314+0000 7ff1c4c00640 1 -- start start 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1c4c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1c00d0e70 0x7ff1c00d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1c4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00b5d40 0x7ff1c00aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1c4c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c00d1790 con 0x7ff1c00b5660 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1c4c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c00727f0 con 0x7ff1c00d0e70 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1c4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4a10 con 0x7ff1c00b5d40 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1bf400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1c00d0e70 0x7ff1c00d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1bf400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff1c00d0e70 0x7ff1c00d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48178/0 (socket says 172.21.15.88:48178) 2024-04-26T20:00:21.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.316+0000 7ff1bf400640 1 -- 172.21.15.88:0/4150744911 learned_addr learned my addr 172.21.15.88:0/4150744911 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.317+0000 7ff1bfe00640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00b5d40 0x7ff1c00aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.317+0000 7ff1bf400640 1 -- 172.21.15.88:0/4150744911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00b5d40 msgr2=0x7ff1c00aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.317+0000 7ff1bf400640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00b5d40 0x7ff1c00aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.317+0000 7ff1bf400640 1 -- 172.21.15.88:0/4150744911 >> v1:172.21.15.132:6789/0 conn(0x7ff1c00b5660 legacy=0x7ff1c00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:21.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.317+0000 7ff1bf400640 1 -- 172.21.15.88:0/4150744911 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00b4cf0 con 0x7ff1c00d0e70 2024-04-26T20:00:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.318+0000 7ff1bf400640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00d0e70 0x7ff1c00d1250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8438bf0745e8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.318+0000 7ff1bea00640 1 -- 172.21.15.88:0/4150744911 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037070 con 0x7ff1c00d0e70 2024-04-26T20:00:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.318+0000 7ff1bea00640 1 -- 172.21.15.88:0/4150744911 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0032440 con 0x7ff1c00d0e70 2024-04-26T20:00:21.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1bea00640 1 -- 172.21.15.88:0/4150744911 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0032810 con 0x7ff1c00d0e70 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 -- 172.21.15.88:0/4150744911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00d0e70 msgr2=0x7ff1c00d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00d0e70 0x7ff1c00d1250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b002a180 comp rx=0 tx=0).stop 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 -- 172.21.15.88:0/4150744911 shutdown_connections 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00b5d40 0x7ff1c00aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/4150744911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00d0e70 0x7ff1c00d1250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.319+0000 7ff1c4c00640 1 -- 172.21.15.88:0/4150744911 >> 172.21.15.88:0/4150744911 conn(0x7ff1c006c740 msgr2=0x7ff1c006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.320+0000 7ff1c4c00640 1 -- 172.21.15.88:0/4150744911 shutdown_connections 2024-04-26T20:00:21.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.320+0000 7ff1c4c00640 1 -- 172.21.15.88:0/4150744911 wait complete. 2024-04-26T20:00:21.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.321+0000 7ff1c4c00640 1 Processor -- start 2024-04-26T20:00:21.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.322+0000 7ff1c4c00640 1 -- start start 2024-04-26T20:00:21.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c00b5d40 0x7ff1c0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00d0e70 0x7ff1c0156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b05b0 con 0x7ff1c00b5660 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b0430 con 0x7ff1c00b5d40 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1c4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c015f1a0 con 0x7ff1c00d0e70 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48194/0 (socket says 172.21.15.88:48194) 2024-04-26T20:00:21.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 -- 172.21.15.88:0/1103572475 learned_addr learned my addr 172.21.15.88:0/1103572475 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 -- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00d0e70 msgr2=0x7ff1c0156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00d0e70 0x7ff1c0156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 -- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c00b5d40 msgr2=0x7ff1c0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c00b5d40 0x7ff1c0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.323+0000 7ff1bf400640 1 -- 172.21.15.88:0/1103572475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c0153720 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1bf400640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0041020 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00306c0 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037040 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c0146080 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.324+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c0146640 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.325+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c00693c0 con 0x7ff1c00b5660 2024-04-26T20:00:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.326+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1b0044020 con 0x7ff1c00b5660 2024-04-26T20:00:21.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.327+0000 7ff1b7e00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 0x7ff19c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.327+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1b00ac360 con 0x7ff1c00b5660 2024-04-26T20:00:21.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.327+0000 7ff1b6a00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 0x7ff19c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.328+0000 7ff1b6a00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 0x7ff19c06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0159730 tx=0x7ff1a8009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.332+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1b004b030 con 0x7ff1c00b5660 2024-04-26T20:00:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.532+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c0159cb0 con 0x7ff1c00b5660 2024-04-26T20:00:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.534+0000 7ff1b7e00640 1 -- 172.21.15.88:0/1103572475 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7ff1b007a780 con 0x7ff1c00b5660 2024-04-26T20:00:21.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:21.535 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 msgr2=0x7ff19c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 0x7ff19c06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0159730 tx=0x7ff1a8009230 comp rx=0 tx=0).stop 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 msgr2=0x7ff1c01445d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009870 tx=0x7ff1b00046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 shutdown_connections 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff19c06c8f0 0x7ff19c06edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1c00d0e70 0x7ff1c0156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1c00b5d40 0x7ff1c0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 --2- 172.21.15.88:0/1103572475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1c00b5660 0x7ff1c01445d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 >> 172.21.15.88:0/1103572475 conn(0x7ff1c006c740 msgr2=0x7ff1c015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 shutdown_connections 2024-04-26T20:00:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.537+0000 7ff1c4c00640 1 -- 172.21.15.88:0/1103572475 wait complete. 2024-04-26T20:00:21.556 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:21.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:21.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.762+0000 7fb94d800640 1 Processor -- start 2024-04-26T20:00:21.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.763+0000 7fb94d800640 1 -- start start 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb94d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 0x7fb948162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb94d800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480d3830 con 0x7fb9480d19c0 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb94d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9480d3b10 con 0x7fb9480b6170 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb94d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb9480d40d0 con 0x7fb9480d1da0 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb946a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb9480d19c0 0x7fb9480d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55708/0 (socket says 172.21.15.88:55708) 2024-04-26T20:00:21.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.764+0000 7fb946a00640 1 -- 172.21.15.88:0/3096175678 learned_addr learned my addr 172.21.15.88:0/3096175678 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb947400640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 0x7fb948162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230232034 0 0) 0x7fb9480d3830 con 0x7fb9480d19c0 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb92c003520 con 0x7fb9480d19c0 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1386173818 0 0) 0x7fb92c003520 con 0x7fb9480d19c0 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 msgr2=0x7fb948162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 0x7fb948162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 >> v1:172.21.15.132:6789/0 conn(0x7fb9480b6170 legacy=0x7fb9480c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480d3df0 con 0x7fb9480d19c0 2024-04-26T20:00:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb938001670 con 0x7fb9480d19c0 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 >> v1:172.21.15.88:6789/0 conn(0x7fb9480d19c0 legacy=0x7fb9480d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb92c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb92c004800 0x7fb92c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 0x7fb92c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c00bac0 con 0x7fb92c0039e0 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c00bc70 con 0x7fb92c004800 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.765+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c00be70 con 0x7fb9480d1da0 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb92c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946000640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb92c004800 0x7fb92c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 -- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 msgr2=0x7fb92c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9480d1da0 0x7fb92c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 -- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb92c004800 msgr2=0x7fb92c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:21.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb92c004800 0x7fb92c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.766+0000 7fb946a00640 1 -- 172.21.15.88:0/3096175678 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c003520 con 0x7fb92c0039e0 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.767+0000 7fb946a00640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb92c003e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c210 tx=0x7fb92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd1fc0c26b0b730 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.767+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938007d30 con 0x7fb92c0039e0 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.767+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb938005af0 con 0x7fb92c0039e0 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.767+0000 7fb945600640 1 -- 172.21.15.88:0/3096175678 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938007070 con 0x7fb92c0039e0 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 -- 172.21.15.88:0/3096175678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 msgr2=0x7fb92c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 --2- 172.21.15.88:0/3096175678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb92c003e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c210 tx=0x7fb92c005830 comp rx=0 tx=0).stop 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb947400640 1 -- 172.21.15.88:0/3096175678 reap_dead start 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 -- 172.21.15.88:0/3096175678 shutdown_connections 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 -- 172.21.15.88:0/3096175678 >> 172.21.15.88:0/3096175678 conn(0x7fb94809e460 msgr2=0x7fb9480d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 -- 172.21.15.88:0/3096175678 shutdown_connections 2024-04-26T20:00:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.768+0000 7fb94d800640 1 -- 172.21.15.88:0/3096175678 wait complete. 2024-04-26T20:00:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.770+0000 7fb94d800640 1 Processor -- start 2024-04-26T20:00:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.770+0000 7fb94d800640 1 -- start start 2024-04-26T20:00:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 0x7fb94814f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94815ba70 0x7fb94815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9481627c0 con 0x7fb92c0039e0 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb948162640 con 0x7fb9480ceca0 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb94d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb948162940 con 0x7fb94815ba70 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb946a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb946a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48212/0 (socket says 172.21.15.88:48212) 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.771+0000 7fb946a00640 1 -- 172.21.15.88:0/3179992683 learned_addr learned my addr 172.21.15.88:0/3179992683 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb947400640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94815ba70 0x7fb94815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946000640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 0x7fb94814f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 -- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94815ba70 msgr2=0x7fb94815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94815ba70 0x7fb94815bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 -- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 msgr2=0x7fb94814f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 0x7fb94814f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 -- 172.21.15.88:0/3179992683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480cf380 con 0x7fb92c0039e0 2024-04-26T20:00:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946000640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 0x7fb94814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.772+0000 7fb946a00640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c300 tx=0x7fb938001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.773+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938042020 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.773+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb938004bf0 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.773+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9480b94c0 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.773+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938007d10 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.773+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb948158a40 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.774+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480b92e0 con 0x7fb92c0039e0 2024-04-26T20:00:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.775+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb938045020 con 0x7fb92c0039e0 2024-04-26T20:00:21.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.776+0000 7fb93ea00640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:21.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.777+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb9380ac4c0 con 0x7fb92c0039e0 2024-04-26T20:00:21.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.777+0000 7fb946000640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:21.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.777+0000 7fb946000640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 0x7fb92406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb930000c00 tx=0x7fb930009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.781+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9480b92e0 con 0x7fb92c0039e0 2024-04-26T20:00:21.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.983+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb948069650 con 0x7fb92c0039e0 2024-04-26T20:00:21.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.984+0000 7fb93ea00640 1 -- 172.21.15.88:0/3179992683 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb93807a840 con 0x7fb92c0039e0 2024-04-26T20:00:21.985 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.986+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 msgr2=0x7fb92406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.986+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 0x7fb92406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb930000c00 tx=0x7fb930009040 comp rx=0 tx=0).stop 2024-04-26T20:00:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.986+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 msgr2=0x7fb948158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.986+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c300 tx=0x7fb938001a40 comp rx=0 tx=0).stop 2024-04-26T20:00:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 shutdown_connections 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92406c8f0 0x7fb92406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94815ba70 0x7fb94815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9480ceca0 0x7fb94814f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 --2- 172.21.15.88:0/3179992683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92c0039e0 0x7fb948158ce0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 >> 172.21.15.88:0/3179992683 conn(0x7fb94809e460 msgr2=0x7fb94806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 shutdown_connections 2024-04-26T20:00:21.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:21.987+0000 7fb94d800640 1 -- 172.21.15.88:0/3179992683 wait complete. 2024-04-26T20:00:22.006 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.208+0000 7f3a92200640 1 Processor -- start 2024-04-26T20:00:22.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.208+0000 7f3a92200640 1 -- start start 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a92200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a8c0b5c20 0x7f3a8c073da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a92200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a8c0ade20 con 0x7f3a8c06ce80 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a92200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a8c0ae100 con 0x7f3a8c147f20 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a92200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8c0742e0 con 0x7f3a8c0b5c20 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a8aa00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3a8c06ce80 0x7f3a8c0aeec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55712/0 (socket says 172.21.15.88:55712) 2024-04-26T20:00:22.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a8aa00640 1 -- 172.21.15.88:0/1621417795 learned_addr learned my addr 172.21.15.88:0/1621417795 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.210+0000 7f3a8be00640 1 --2- 172.21.15.88:0/1621417795 >> v2:172.21.15.132:3300/0 conn(0x7f3a8c0b5c20 0x7f3a8c073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852944221 0 0) 0x7f3a8c0ade20 con 0x7f3a8c06ce80 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a70003520 con 0x7f3a8c06ce80 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4175976715 0 0) 0x7f3a70003520 con 0x7f3a8c06ce80 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 >> v2:172.21.15.132:3300/0 conn(0x7f3a8c0b5c20 msgr2=0x7f3a8c073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 --2- 172.21.15.88:0/1621417795 >> v2:172.21.15.132:3300/0 conn(0x7f3a8c0b5c20 0x7f3a8c073da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 >> v1:172.21.15.132:6789/0 conn(0x7f3a8c147f20 legacy=0x7f3a8c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c0ae3e0 con 0x7f3a8c06ce80 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f3a78001670 con 0x7f3a8c06ce80 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 >> v1:172.21.15.88:6789/0 conn(0x7f3a8c06ce80 legacy=0x7f3a8c0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a70003a00 0x7f3a70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 0x7f3a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a700051c0 0x7f3a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000c090 con 0x7f3a70004800 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7000c240 con 0x7f3a70003a00 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7000c440 con 0x7f3a700051c0 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.211+0000 7f3a8aa00640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 0x7f3a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8b400640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a70003a00 0x7f3a70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8be00640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a700051c0 0x7f3a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8aa00640 1 -- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a700051c0 msgr2=0x7f3a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8aa00640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a700051c0 0x7f3a7000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8aa00640 1 -- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a70003a00 msgr2=0x7f3a70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8aa00640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a70003a00 0x7f3a70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8aa00640 1 -- 172.21.15.88:0/1621417795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a70003520 con 0x7f3a70004800 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8be00640 1 -- 172.21.15.88:0/1621417795 reap_dead start 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.212+0000 7f3a8b400640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a70003a00 0x7f3a70003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.213+0000 7f3a8aa00640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 0x7f3a70004c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a78007fd0 tx=0x7f3a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8542f079c72d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.213+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78007c20 con 0x7f3a70004800 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.213+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78005af0 con 0x7f3a70004800 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.213+0000 7f3a8a000640 1 -- 172.21.15.88:0/1621417795 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78007070 con 0x7f3a70004800 2024-04-26T20:00:22.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 -- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 msgr2=0x7f3a70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 0x7f3a70004c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a78007fd0 tx=0x7f3a70005830 comp rx=0 tx=0).stop 2024-04-26T20:00:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 -- 172.21.15.88:0/1621417795 shutdown_connections 2024-04-26T20:00:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 --2- 172.21.15.88:0/1621417795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a70004800 0x7f3a70004c80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 -- 172.21.15.88:0/1621417795 >> 172.21.15.88:0/1621417795 conn(0x7f3a8c0b2d50 msgr2=0x7f3a8c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 -- 172.21.15.88:0/1621417795 shutdown_connections 2024-04-26T20:00:22.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.214+0000 7f3a92200640 1 -- 172.21.15.88:0/1621417795 wait complete. 2024-04-26T20:00:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.216+0000 7f3a92200640 1 Processor -- start 2024-04-26T20:00:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.216+0000 7f3a92200640 1 -- start start 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a70004800 0x7f3a8c141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c142550 0x7f3a8c1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c0b6050 con 0x7f3a8c160b10 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c0ae9a0 con 0x7f3a8c142550 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a92200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c06e220 con 0x7f3a70004800 2024-04-26T20:00:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48230/0 (socket says 172.21.15.88:48230) 2024-04-26T20:00:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 -- 172.21.15.88:0/2345217064 learned_addr learned my addr 172.21.15.88:0/2345217064 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8b400640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a70004800 0x7f3a8c141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8aa00640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c142550 0x7f3a8c1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 -- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a70004800 msgr2=0x7f3a8c141fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a70004800 0x7f3a8c141fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 -- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c142550 msgr2=0x7f3a8c1605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.217+0000 7f3a8be00640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c142550 0x7f3a8c1605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a8be00640 1 -- 172.21.15.88:0/2345217064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c142dd0 con 0x7f3a8c160b10 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a8be00640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a8400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84016420 con 0x7f3a8c160b10 2024-04-26T20:00:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8401b070 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84016800 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c0cfd70 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.218+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c0d0330 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.219+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c069650 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.220+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a840169a0 con 0x7f3a8c160b10 2024-04-26T20:00:22.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.222+0000 7f3a83400640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 0x7f3a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.222+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a8408c730 con 0x7f3a8c160b10 2024-04-26T20:00:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.222+0000 7f3a8b400640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 0x7f3a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.223+0000 7f3a8b400640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 0x7f3a6c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.227+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a84024020 con 0x7f3a8c160b10 2024-04-26T20:00:22.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.426+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a8c0b4540 con 0x7f3a8c160b10 2024-04-26T20:00:22.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.427+0000 7f3a83400640 1 -- 172.21.15.88:0/2345217064 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3a8c142dd0 con 0x7f3a8c160b10 2024-04-26T20:00:22.428 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 msgr2=0x7f3a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 0x7f3a6c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c009040 comp rx=0 tx=0).stop 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 msgr2=0x7f3a8c09ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009fd0 tx=0x7f3a8400f520 comp rx=0 tx=0).stop 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 shutdown_connections 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a8c160b10 0x7f3a8c09ba50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a8c142550 0x7f3a8c1605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6c06c8f0 0x7f3a6c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 --2- 172.21.15.88:0/2345217064 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a70004800 0x7f3a8c141fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 >> 172.21.15.88:0/2345217064 conn(0x7f3a8c0b2d50 msgr2=0x7f3a8c147060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 shutdown_connections 2024-04-26T20:00:22.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.430+0000 7f3a92200640 1 -- 172.21.15.88:0/2345217064 wait complete. 2024-04-26T20:00:22.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.650+0000 7f8d59800640 1 Processor -- start 2024-04-26T20:00:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.651+0000 7f8d59800640 1 -- start start 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8d540d19c0 0x7f8d540d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8d540b6170 0x7f8d540c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 0x7f8d54162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8d540d40d0 con 0x7f8d540b6170 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8d540d3830 con 0x7f8d540d19c0 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d59800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8d540d3b10 con 0x7f8d540d1da0 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.652+0000 7f8d52000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8d540b6170 0x7f8d540c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8d540b6170 0x7f8d540c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48246/0 (socket says 172.21.15.88:48246) 2024-04-26T20:00:22.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 -- 172.21.15.88:0/2855523216 learned_addr learned my addr 172.21.15.88:0/2855523216 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52a00640 1 --2- 172.21.15.88:0/2855523216 >> v2:172.21.15.132:3300/0 conn(0x7f8d540d19c0 0x7f8d540d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d53400640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 0x7f8d54162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 -- 172.21.15.88:0/2855523216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 msgr2=0x7f8d54162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 0x7f8d54162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 -- 172.21.15.88:0/2855523216 >> v2:172.21.15.132:3300/0 conn(0x7f8d540d19c0 msgr2=0x7f8d540d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 --2- 172.21.15.88:0/2855523216 >> v2:172.21.15.132:3300/0 conn(0x7f8d540d19c0 0x7f8d540d2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d52000640 1 -- 172.21.15.88:0/2855523216 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d540d3df0 con 0x7f8d540b6170 2024-04-26T20:00:22.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.653+0000 7f8d53400640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 0x7f8d54162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.654+0000 7f8d52000640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540b6170 0x7f8d540c8c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d5406ca80 tx=0x7f8d4402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f62a567cd43fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.654+0000 7f8d51600640 1 -- 172.21.15.88:0/2855523216 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44036070 con 0x7f8d540b6170 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.654+0000 7f8d51600640 1 -- 172.21.15.88:0/2855523216 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d44031600 con 0x7f8d540b6170 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.654+0000 7f8d51600640 1 -- 172.21.15.88:0/2855523216 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440319d0 con 0x7f8d540b6170 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 -- 172.21.15.88:0/2855523216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540b6170 msgr2=0x7f8d540c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540b6170 0x7f8d540c8c70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8d5406ca80 tx=0x7f8d4402a610 comp rx=0 tx=0).stop 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 -- 172.21.15.88:0/2855523216 shutdown_connections 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540d1da0 0x7f8d54162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 --2- 172.21.15.88:0/2855523216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540b6170 0x7f8d540c8c70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 --2- 172.21.15.88:0/2855523216 >> v2:172.21.15.132:3300/0 conn(0x7f8d540d19c0 0x7f8d540d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 -- 172.21.15.88:0/2855523216 >> 172.21.15.88:0/2855523216 conn(0x7f8d5409e460 msgr2=0x7f8d540d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 -- 172.21.15.88:0/2855523216 shutdown_connections 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.655+0000 7f8d59800640 1 -- 172.21.15.88:0/2855523216 wait complete. 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.657+0000 7f8d59800640 1 Processor -- start 2024-04-26T20:00:22.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.657+0000 7f8d59800640 1 -- start start 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540b6170 0x7f8d541e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d540d1da0 0x7f8d541eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540cfc30 con 0x7f8d540d19c0 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540c91b0 con 0x7f8d540d1da0 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d59800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8d54162b80 con 0x7f8d540b6170 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d52000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d52000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48248/0 (socket says 172.21.15.88:48248) 2024-04-26T20:00:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.658+0000 7f8d52000640 1 -- 172.21.15.88:0/3501165380 learned_addr learned my addr 172.21.15.88:0/3501165380 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:22.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52a00640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540b6170 0x7f8d541e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 -- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540b6170 msgr2=0x7f8d541e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540b6170 0x7f8d541e7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 -- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d540d1da0 msgr2=0x7f8d541eabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d540d1da0 0x7f8d541eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 -- 172.21.15.88:0/3501165380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d54149220 con 0x7f8d540d19c0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d52000640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f80 tx=0x7f8d44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44036050 con 0x7f8d540d19c0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d44042070 con 0x7f8d540d19c0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44009510 con 0x7f8d540d19c0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.659+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541494c0 con 0x7f8d540d19c0 2024-04-26T20:00:22.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.660+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d54149a80 con 0x7f8d540d19c0 2024-04-26T20:00:22.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.661+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d54069650 con 0x7f8d540d19c0 2024-04-26T20:00:22.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.662+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d44007c70 con 0x7f8d540d19c0 2024-04-26T20:00:22.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.663+0000 7f8d4aa00640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 0x7f8d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:22.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.663+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d440aca10 con 0x7f8d540d19c0 2024-04-26T20:00:22.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.663+0000 7f8d52a00640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 0x7f8d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:22.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.664+0000 7f8d52a00640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 0x7f8d3006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c006fd0 tx=0x7f8d3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.667+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d44045360 con 0x7f8d540d19c0 2024-04-26T20:00:22.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.867+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d540d26d0 con 0x7f8d540d19c0 2024-04-26T20:00:22.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.868+0000 7f8d4aa00640 1 -- 172.21.15.88:0/3501165380 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8d4407ae40 con 0x7f8d540d19c0 2024-04-26T20:00:22.870 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 msgr2=0x7f8d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 0x7f8d3006edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c006fd0 tx=0x7f8d3c008040 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 msgr2=0x7f8d541ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8d44000f80 tx=0x7f8d44004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 shutdown_connections 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d540d1da0 0x7f8d541eabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d540d19c0 0x7f8d541ea680 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8d3006c8f0 0x7f8d3006edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 --2- 172.21.15.88:0/3501165380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8d540b6170 0x7f8d541e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 >> 172.21.15.88:0/3501165380 conn(0x7f8d5409e460 msgr2=0x7f8d541619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 shutdown_connections 2024-04-26T20:00:22.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:22.871+0000 7f8d59800640 1 -- 172.21.15.88:0/3501165380 wait complete. 2024-04-26T20:00:22.889 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:23.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.085+0000 7f7a5be00640 1 Processor -- start 2024-04-26T20:00:23.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.086+0000 7f7a5be00640 1 -- start start 2024-04-26T20:00:23.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7a5c145570 0x7f7a5c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7a5c073ed0 0x7f7a5c0d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5be00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5c073110 con 0x7f7a5c15f940 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5be00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0cef40 con 0x7f7a5c073ed0 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c1606b0 con 0x7f7a5c145570 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5a000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7a5c073ed0 0x7f7a5c0d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7a5c15f940 0x7f7a5c15fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55724/0 (socket says 172.21.15.88:55724) 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.088+0000 7f7a5b400640 1 -- 172.21.15.88:0/3003402549 learned_addr learned my addr 172.21.15.88:0/3003402549 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.089+0000 7f7a5a000640 1 -- 172.21.15.88:0/3003402549 >> v2:172.21.15.132:3300/0 conn(0x7f7a5c145570 msgr2=0x7f7a5c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.089+0000 7f7a5a000640 1 --2- 172.21.15.88:0/3003402549 >> v2:172.21.15.132:3300/0 conn(0x7f7a5c145570 0x7f7a5c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.089+0000 7f7a5a000640 1 -- 172.21.15.88:0/3003402549 >> v1:172.21.15.88:6789/0 conn(0x7f7a5c15f940 legacy=0x7f7a5c15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.089+0000 7f7a5a000640 1 -- 172.21.15.88:0/3003402549 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c0733f0 con 0x7f7a5c073ed0 2024-04-26T20:00:23.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.089+0000 7f7a59600640 1 -- 172.21.15.88:0/3003402549 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851814120 0 0) 0x7f7a5c073110 con 0x7f7a5c15f940 2024-04-26T20:00:23.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.090+0000 7f7a5a000640 1 --2- 172.21.15.88:0/3003402549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c073ed0 0x7f7a5c0d13b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0d1ab0 tx=0x7f7a4802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35291d0767e8c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.092+0000 7f7a59600640 1 -- 172.21.15.88:0/3003402549 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48037070 con 0x7f7a5c073ed0 2024-04-26T20:00:23.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.092+0000 7f7a59600640 1 -- 172.21.15.88:0/3003402549 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a48032440 con 0x7f7a5c073ed0 2024-04-26T20:00:23.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.092+0000 7f7a59600640 1 -- 172.21.15.88:0/3003402549 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4802ae60 con 0x7f7a5c073ed0 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 -- 172.21.15.88:0/3003402549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c073ed0 msgr2=0x7f7a5c0d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 --2- 172.21.15.88:0/3003402549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c073ed0 0x7f7a5c0d13b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0d1ab0 tx=0x7f7a4802a110 comp rx=0 tx=0).stop 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 -- 172.21.15.88:0/3003402549 shutdown_connections 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 --2- 172.21.15.88:0/3003402549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c073ed0 0x7f7a5c0d13b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 --2- 172.21.15.88:0/3003402549 >> v2:172.21.15.132:3300/0 conn(0x7f7a5c145570 0x7f7a5c0d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 -- 172.21.15.88:0/3003402549 >> 172.21.15.88:0/3003402549 conn(0x7f7a5c06c740 msgr2=0x7f7a5c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:23.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 -- 172.21.15.88:0/3003402549 shutdown_connections 2024-04-26T20:00:23.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.093+0000 7f7a5be00640 1 -- 172.21.15.88:0/3003402549 wait complete. 2024-04-26T20:00:23.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.095+0000 7f7a5be00640 1 Processor -- start 2024-04-26T20:00:23.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.095+0000 7f7a5be00640 1 -- start start 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 0x7f7a5c140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 0x7f7a5c1e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c145950 con 0x7f7a5c15f940 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0739b0 con 0x7f7a5c145570 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c0cdf30 con 0x7f7a5c073ed0 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.096+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48260/0 (socket says 172.21.15.88:48260) 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 -- 172.21.15.88:0/873747811 learned_addr learned my addr 172.21.15.88:0/873747811 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:23.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5a000640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 0x7f7a5c1e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5aa00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 0x7f7a5c140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 -- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 msgr2=0x7f7a5c140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 0x7f7a5c140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 -- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 msgr2=0x7f7a5c1e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 0x7f7a5c1e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 -- 172.21.15.88:0/873747811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c140960 con 0x7f7a5c15f940 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5aa00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 0x7f7a5c140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5a000640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 0x7f7a5c1e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.097+0000 7f7a5b400640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c700 tx=0x7f7a5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.098+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54018070 con 0x7f7a5c15f940 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.098+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5400ce10 con 0x7f7a5c15f940 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.098+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400a860 con 0x7f7a5c15f940 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.098+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c1e66c0 con 0x7f7a5c15f940 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.098+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1e6c80 con 0x7f7a5c15f940 2024-04-26T20:00:23.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.099+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c0693c0 con 0x7f7a5c15f940 2024-04-26T20:00:23.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.100+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a5401b070 con 0x7f7a5c15f940 2024-04-26T20:00:23.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.101+0000 7f7a53400640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 0x7f7a3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.101+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7a5408b160 con 0x7f7a5c15f940 2024-04-26T20:00:23.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.101+0000 7f7a5aa00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 0x7f7a3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.102+0000 7f7a5aa00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 0x7f7a3c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006fd0 tx=0x7f7a4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.106+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a540594f0 con 0x7f7a5c15f940 2024-04-26T20:00:23.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.305+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a5c0b4850 con 0x7f7a5c15f940 2024-04-26T20:00:23.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.307+0000 7f7a53400640 1 -- 172.21.15.88:0/873747811 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f7a54058df0 con 0x7f7a5c15f940 2024-04-26T20:00:23.308 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:23.308 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 msgr2=0x7f7a3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 0x7f7a3c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006fd0 tx=0x7f7a4c008040 comp rx=0 tx=0).stop 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 msgr2=0x7f7a5c0cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c700 tx=0x7f7a5400cbc0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 shutdown_connections 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a5c15f940 0x7f7a5c0cf0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7a5c145570 0x7f7a5c1e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7a3c06c8f0 0x7f7a3c06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 --2- 172.21.15.88:0/873747811 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7a5c073ed0 0x7f7a5c140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 >> 172.21.15.88:0/873747811 conn(0x7f7a5c06c740 msgr2=0x7f7a5c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:23.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 shutdown_connections 2024-04-26T20:00:23.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.310+0000 7f7a5be00640 1 -- 172.21.15.88:0/873747811 wait complete. 2024-04-26T20:00:23.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:23.331 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:23.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.532+0000 7f8143200640 1 Processor -- start 2024-04-26T20:00:23.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.533+0000 7f8143200640 1 -- start start 2024-04-26T20:00:23.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f8143200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f814415f7f0 0x7f81440cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f8143200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f814415fbd0 0x7f81440d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f8143200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440d0170 con 0x7f8144162cf0 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f8143200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f81440d0450 con 0x7f814415fbd0 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f8143200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f81440d0730 con 0x7f814415f7f0 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f813be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f814415fbd0 0x7f81440d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f813be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f814415fbd0 0x7f81440d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48268/0 (socket says 172.21.15.88:48268) 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.534+0000 7f813be00640 1 -- 172.21.15.88:0/1113702066 learned_addr learned my addr 172.21.15.88:0/1113702066 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.535+0000 7f813be00640 1 -- 172.21.15.88:0/1113702066 >> v2:172.21.15.132:3300/0 conn(0x7f814415f7f0 msgr2=0x7f81440cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.535+0000 7f813be00640 1 --2- 172.21.15.88:0/1113702066 >> v2:172.21.15.132:3300/0 conn(0x7f814415f7f0 0x7f81440cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.535+0000 7f813be00640 1 -- 172.21.15.88:0/1113702066 >> v1:172.21.15.132:6789/0 conn(0x7f8144162cf0 legacy=0x7f8144163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.535+0000 7f813be00640 1 -- 172.21.15.88:0/1113702066 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440d0a10 con 0x7f814415fbd0 2024-04-26T20:00:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.536+0000 7f813be00640 1 --2- 172.21.15.88:0/1113702066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f814415fbd0 0x7f81440d22d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f814408dbd0 tx=0x7f812c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d363d6af58815 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.536+0000 7f8141400640 1 -- 172.21.15.88:0/1113702066 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c037070 con 0x7f814415fbd0 2024-04-26T20:00:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.536+0000 7f8141400640 1 -- 172.21.15.88:0/1113702066 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812c032440 con 0x7f814415fbd0 2024-04-26T20:00:23.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.536+0000 7f8141400640 1 -- 172.21.15.88:0/1113702066 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c032810 con 0x7f814415fbd0 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 -- 172.21.15.88:0/1113702066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f814415fbd0 msgr2=0x7f81440d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 --2- 172.21.15.88:0/1113702066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f814415fbd0 0x7f81440d22d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f814408dbd0 tx=0x7f812c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 -- 172.21.15.88:0/1113702066 shutdown_connections 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 --2- 172.21.15.88:0/1113702066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f814415fbd0 0x7f81440d22d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 --2- 172.21.15.88:0/1113702066 >> v2:172.21.15.132:3300/0 conn(0x7f814415f7f0 0x7f81440cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 -- 172.21.15.88:0/1113702066 >> 172.21.15.88:0/1113702066 conn(0x7f81440d4f80 msgr2=0x7f81440d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 -- 172.21.15.88:0/1113702066 shutdown_connections 2024-04-26T20:00:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.537+0000 7f8143200640 1 -- 172.21.15.88:0/1113702066 wait complete. 2024-04-26T20:00:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.539+0000 7f8143200640 1 Processor -- start 2024-04-26T20:00:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.539+0000 7f8143200640 1 -- start start 2024-04-26T20:00:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 0x7f81441f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 0x7f81440ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8144160050 con 0x7f8144162cf0 2024-04-26T20:00:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f81440d2810 con 0x7f814415fbd0 2024-04-26T20:00:23.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8143200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f814415f430 con 0x7f814415f7f0 2024-04-26T20:00:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8142800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8142800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48272/0 (socket says 172.21.15.88:48272) 2024-04-26T20:00:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8142800640 1 -- 172.21.15.88:0/1783978348 learned_addr learned my addr 172.21.15.88:0/1783978348 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f813be00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 0x7f81440ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.540+0000 7f8141e00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 0x7f81441f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 -- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 msgr2=0x7f81441f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 0x7f81441f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 -- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 msgr2=0x7f81440ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 0x7f81440ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 -- 172.21.15.88:0/1783978348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440ba320 con 0x7f8144162cf0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f813be00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 0x7f81440ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8141e00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 0x7f81441f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8142800640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c780 tx=0x7f813c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c017070 con 0x7f8144162cf0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c003da0 con 0x7f8144162cf0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0052b0 con 0x7f8144162cf0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81440ba600 con 0x7f8144162cf0 2024-04-26T20:00:23.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.541+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81440cc0f0 con 0x7f8144162cf0 2024-04-26T20:00:23.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.542+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81440693c0 con 0x7f8144162cf0 2024-04-26T20:00:23.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.543+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f813c006b10 con 0x7f8144162cf0 2024-04-26T20:00:23.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.544+0000 7f813a000640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 0x7f812006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.544+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f813c08b1f0 con 0x7f8144162cf0 2024-04-26T20:00:23.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.545+0000 7f8141e00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 0x7f812006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.545+0000 7f8141e00640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 0x7f812006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81441f8b40 tx=0x7f8134008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.549+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f813c059580 con 0x7f8144162cf0 2024-04-26T20:00:23.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.751+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81440d4240 con 0x7f8144162cf0 2024-04-26T20:00:23.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.752+0000 7f813a000640 1 -- 172.21.15.88:0/1783978348 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f813c058e80 con 0x7f8144162cf0 2024-04-26T20:00:23.753 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:23.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.754+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 msgr2=0x7f812006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.754+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 0x7f812006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f81441f8b40 tx=0x7f8134008040 comp rx=0 tx=0).stop 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.754+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 msgr2=0x7f81440c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.754+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f813c00c780 tx=0x7f813c00cc40 comp rx=0 tx=0).stop 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 shutdown_connections 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8144162cf0 0x7f81440c9a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f814415fbd0 0x7f81440ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f812006c8f0 0x7f812006edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 --2- 172.21.15.88:0/1783978348 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f814415f7f0 0x7f81441f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 >> 172.21.15.88:0/1783978348 conn(0x7f81440d4f80 msgr2=0x7f814406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 shutdown_connections 2024-04-26T20:00:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.755+0000 7f8143200640 1 -- 172.21.15.88:0/1783978348 wait complete. 2024-04-26T20:00:23.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:23.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.981+0000 7f8540e00640 1 Processor -- start 2024-04-26T20:00:23.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.982+0000 7f8540e00640 1 -- start start 2024-04-26T20:00:23.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.983+0000 7f8540e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f853c06e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.983+0000 7f8540e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c163c20 con 0x7f853c162fc0 2024-04-26T20:00:23.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.983+0000 7f8540e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f853c06d880 con 0x7f853c0cf780 2024-04-26T20:00:23.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.983+0000 7f8540e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f853c163f00 con 0x7f853c15f460 2024-04-26T20:00:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.984+0000 7f853a800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f853c162fc0 0x7f853c163430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55736/0 (socket says 172.21.15.88:55736) 2024-04-26T20:00:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.984+0000 7f853a800640 1 -- 172.21.15.88:0/4005364693 learned_addr learned my addr 172.21.15.88:0/4005364693 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.984+0000 7f8539e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f853c06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.984+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3761332110 0 0) 0x7f853c163c20 con 0x7f853c162fc0 2024-04-26T20:00:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.984+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8520003520 con 0x7f853c162fc0 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2800689003 0 0) 0x7f8520003520 con 0x7f853c162fc0 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 msgr2=0x7f853c06e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f853c06e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 >> v1:172.21.15.132:6789/0 conn(0x7f853c0cf780 legacy=0x7f853c0cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c06db60 con 0x7f853c162fc0 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f85340019f0 con 0x7f853c162fc0 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 >> v1:172.21.15.88:6789/0 conn(0x7f853c162fc0 legacy=0x7f853c163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85200039e0 0x7f8520003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8520004800 0x7f8520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f8520005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f853c163f00 con 0x7f8520004800 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f852000bb50 con 0x7f85200039e0 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f852000bd50 con 0x7f853c15f460 2024-04-26T20:00:23.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8533e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8520004800 0x7f8520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.985+0000 7f8539e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85200039e0 0x7f8520003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f853a800640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f8520005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f8533e00640 1 -- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 msgr2=0x7f8520005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f8533e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f8520005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f8533e00640 1 -- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85200039e0 msgr2=0x7f8520003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f8533e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85200039e0 0x7f8520003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f8533e00640 1 -- 172.21.15.88:0/4005364693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8520003520 con 0x7f8520004800 2024-04-26T20:00:23.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.986+0000 7f853a800640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f853c15f460 0x7f8520005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:23.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8533e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8520004800 0x7f8520004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8524001750 tx=0x7f8520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d37e827ee3267d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524001370 con 0x7f8520004800 2024-04-26T20:00:23.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85240052d0 con 0x7f8520004800 2024-04-26T20:00:23.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8539400640 1 -- 172.21.15.88:0/4005364693 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852402abe0 con 0x7f8520004800 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8540e00640 1 -- 172.21.15.88:0/4005364693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8520004800 msgr2=0x7f8520004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f8540e00640 1 --2- 172.21.15.88:0/4005364693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8520004800 0x7f8520004c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8524001750 tx=0x7f8520005830 comp rx=0 tx=0).stop 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.987+0000 7f853a800640 1 -- 172.21.15.88:0/4005364693 reap_dead start 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.988+0000 7f8540e00640 1 -- 172.21.15.88:0/4005364693 shutdown_connections 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.988+0000 7f8540e00640 1 -- 172.21.15.88:0/4005364693 >> 172.21.15.88:0/4005364693 conn(0x7f853c0b4a50 msgr2=0x7f853c0b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.988+0000 7f8540e00640 1 -- 172.21.15.88:0/4005364693 shutdown_connections 2024-04-26T20:00:23.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.988+0000 7f8540e00640 1 -- 172.21.15.88:0/4005364693 wait complete. 2024-04-26T20:00:23.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.989+0000 7f8540e00640 1 Processor -- start 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.990+0000 7f8540e00640 1 -- start start 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.990+0000 7f8540e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8520004800 0x7f853c0b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8540e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8540e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f853c0c8630 0x7f853c0c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8540e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f853c06e360 con 0x7f853c0c6a30 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8540e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f853c15f840 con 0x7f853c0c8630 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8540e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f853c06e4e0 con 0x7f8520004800 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48296/0 (socket says 172.21.15.88:48296) 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 -- 172.21.15.88:0/3692704590 learned_addr learned my addr 172.21.15.88:0/3692704590 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 -- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8520004800 msgr2=0x7f853c0b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f853a800640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f853c0c8630 0x7f853c0c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8539e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8520004800 0x7f853c0b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8520004800 0x7f853c0b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 -- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f853c0c8630 msgr2=0x7f853c0c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f853c0c8630 0x7f853c0c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.991+0000 7f8533e00640 1 -- 172.21.15.88:0/3692704590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c15fdc0 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8533e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8524001840 tx=0x7f8524001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524037040 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8524042070 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524007a30 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1600a0 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.992+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c0c8ff0 con 0x7f853c0c6a30 2024-04-26T20:00:23.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.993+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f853c0b8e00 con 0x7f853c0c6a30 2024-04-26T20:00:23.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.994+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85240052d0 con 0x7f853c0c6a30 2024-04-26T20:00:23.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.995+0000 7f8532000640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:23.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.995+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85240abe00 con 0x7f853c0c6a30 2024-04-26T20:00:23.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.995+0000 7f8539e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:23.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.996+0000 7f8539e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 0x7f851806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f853c1f87b0 tx=0x7f852c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:23.999+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f853c0b8e00 con 0x7f853c0c6a30 2024-04-26T20:00:24.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.200+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f853c069650 con 0x7f853c0c6a30 2024-04-26T20:00:24.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.202+0000 7f8532000640 1 -- 172.21.15.88:0/3692704590 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f852407a2c0 con 0x7f853c0c6a30 2024-04-26T20:00:24.202 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 msgr2=0x7f851806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 0x7f851806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f853c1f87b0 tx=0x7f852c008040 comp rx=0 tx=0).stop 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 msgr2=0x7f853c0c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8524001840 tx=0x7f8524001120 comp rx=0 tx=0).stop 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 shutdown_connections 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f853c0c8630 0x7f853c0c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f853c0c6a30 0x7f853c0c93f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f851806c8f0 0x7f851806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 --2- 172.21.15.88:0/3692704590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8520004800 0x7f853c0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.204+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 >> 172.21.15.88:0/3692704590 conn(0x7f853c0b4a50 msgr2=0x7f853c15d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.205+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 shutdown_connections 2024-04-26T20:00:24.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.205+0000 7f8540e00640 1 -- 172.21.15.88:0/3692704590 wait complete. 2024-04-26T20:00:24.225 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:24.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.423+0000 7f1487200640 1 Processor -- start 2024-04-26T20:00:24.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.423+0000 7f1487200640 1 -- start start 2024-04-26T20:00:24.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1487200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1488156370 0x7f148815c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1487200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f148815c760 0x7f14880d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1487200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1488156750 con 0x7f14880d12a0 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1487200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f148815d430 con 0x7f1488156370 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1487200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f14880d16d0 con 0x7f148815c760 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1486800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f14880d12a0 0x7f148815f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55740/0 (socket says 172.21.15.88:55740) 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1486800640 1 -- 172.21.15.88:0/3473726487 learned_addr learned my addr 172.21.15.88:0/3473726487 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1485e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1488156370 0x7f148815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1485400640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f148815c760 0x7f14880d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.425+0000 7f1484a00640 1 -- 172.21.15.88:0/3473726487 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904395016 0 0) 0x7f1480001490 con 0x7f14880d12a0 2024-04-26T20:00:24.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.426+0000 7f1484a00640 1 -- 172.21.15.88:0/3473726487 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f146c003520 con 0x7f14880d12a0 2024-04-26T20:00:24.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.426+0000 7f1485e00640 1 -- 172.21.15.88:0/3473726487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f148815c760 msgr2=0x7f14880d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:24.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.426+0000 7f1485e00640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f148815c760 0x7f14880d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.426+0000 7f1485e00640 1 -- 172.21.15.88:0/3473726487 >> v1:172.21.15.88:6789/0 conn(0x7f14880d12a0 legacy=0x7f148815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.426+0000 7f1485e00640 1 -- 172.21.15.88:0/3473726487 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148815cd60 con 0x7f1488156370 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.427+0000 7f1485e00640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f148815c220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14780097a0 tx=0x7f147802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90b284cb2dbd842 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.427+0000 7f1484a00640 1 -- 172.21.15.88:0/3473726487 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478037070 con 0x7f1488156370 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.427+0000 7f1484a00640 1 -- 172.21.15.88:0/3473726487 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1478032440 con 0x7f1488156370 2024-04-26T20:00:24.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.427+0000 7f1484a00640 1 -- 172.21.15.88:0/3473726487 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478032810 con 0x7f1488156370 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 -- 172.21.15.88:0/3473726487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 msgr2=0x7f148815c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f148815c220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f14780097a0 tx=0x7f147802a180 comp rx=0 tx=0).stop 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 -- 172.21.15.88:0/3473726487 shutdown_connections 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f148815c760 0x7f14880d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 --2- 172.21.15.88:0/3473726487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f148815c220 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 -- 172.21.15.88:0/3473726487 >> 172.21.15.88:0/3473726487 conn(0x7f148806c530 msgr2=0x7f148806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 -- 172.21.15.88:0/3473726487 shutdown_connections 2024-04-26T20:00:24.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.428+0000 7f1487200640 1 -- 172.21.15.88:0/3473726487 wait complete. 2024-04-26T20:00:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.429+0000 7f1487200640 1 Processor -- start 2024-04-26T20:00:24.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.430+0000 7f1487200640 1 -- start start 2024-04-26T20:00:24.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.430+0000 7f1487200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14880d12a0 0x7f148813b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1487200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1487200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f148815c760 0x7f1488154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1487200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f148815d960 con 0x7f1488156370 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1487200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f148815d680 con 0x7f148815c760 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1487200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f14880b3cb0 con 0x7f14880d12a0 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48308/0 (socket says 172.21.15.88:48308) 2024-04-26T20:00:24.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 -- 172.21.15.88:0/872238808 learned_addr learned my addr 172.21.15.88:0/872238808 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485e00640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14880d12a0 0x7f148813b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 -- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14880d12a0 msgr2=0x7f148813b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14880d12a0 0x7f148813b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 -- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f148815c760 msgr2=0x7f1488154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f148815c760 0x7f1488154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.431+0000 7f1485400640 1 -- 172.21.15.88:0/872238808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148813baa0 con 0x7f1488156370 2024-04-26T20:00:24.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f1485400640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f147400d5f0 tx=0x7f147400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14740125f0 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1474019070 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14740129d0 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1488153f50 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.432+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1488154510 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.433+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14880693c0 con 0x7f1488156370 2024-04-26T20:00:24.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.434+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1474012b70 con 0x7f1488156370 2024-04-26T20:00:24.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.435+0000 7f147e000640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 0x7f146806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.436+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f147408bfe0 con 0x7f1488156370 2024-04-26T20:00:24.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.436+0000 7f1485e00640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 0x7f146806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.437+0000 7f1485e00640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 0x7f146806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f147802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.440+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f147405a3d0 con 0x7f1488156370 2024-04-26T20:00:24.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.640+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f148806fba0 con 0x7f1488156370 2024-04-26T20:00:24.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.641+0000 7f147e000640 1 -- 172.21.15.88:0/872238808 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1474059cd0 con 0x7f1488156370 2024-04-26T20:00:24.641 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 msgr2=0x7f146806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 0x7f146806edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f147802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 msgr2=0x7f1488072860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f147400d5f0 tx=0x7f147400dab0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 shutdown_connections 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f148815c760 0x7f1488154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1488156370 0x7f1488072860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f146806c8f0 0x7f146806edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 --2- 172.21.15.88:0/872238808 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f14880d12a0 0x7f148813b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 >> 172.21.15.88:0/872238808 conn(0x7f148806c530 msgr2=0x7f148815f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:24.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 shutdown_connections 2024-04-26T20:00:24.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.643+0000 7f1487200640 1 -- 172.21.15.88:0/872238808 wait complete. 2024-04-26T20:00:24.663 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:24.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.864+0000 7fa6a3e00640 1 Processor -- start 2024-04-26T20:00:24.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.864+0000 7fa6a3e00640 1 -- start start 2024-04-26T20:00:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.865+0000 7fa6a3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6a4146680 0x7fa6a40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.865+0000 7fa6a3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa6a40d0e70 0x7fa6a40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.865+0000 7fa6a3e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a4146a60 con 0x7fa6a4160930 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.865+0000 7fa6a3e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a406e970 con 0x7fa6a40d0e70 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.865+0000 7fa6a3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a40d17f0 con 0x7fa6a4146680 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa6a40d0e70 0x7fa6a40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa6a40d0e70 0x7fa6a40d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48312/0 (socket says 172.21.15.88:48312) 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 -- 172.21.15.88:0/2837785510 learned_addr learned my addr 172.21.15.88:0/2837785510 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa6a2a00640 1 --2- 172.21.15.88:0/2837785510 >> v2:172.21.15.132:3300/0 conn(0x7fa6a4146680 0x7fa6a40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 -- 172.21.15.88:0/2837785510 >> v2:172.21.15.132:3300/0 conn(0x7fa6a4146680 msgr2=0x7fa6a40b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 --2- 172.21.15.88:0/2837785510 >> v2:172.21.15.132:3300/0 conn(0x7fa6a4146680 0x7fa6a40b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 -- 172.21.15.88:0/2837785510 >> v1:172.21.15.132:6789/0 conn(0x7fa6a4160930 legacy=0x7fa6a4160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.866+0000 7fa69be00640 1 -- 172.21.15.88:0/2837785510 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a406dd60 con 0x7fa6a40d0e70 2024-04-26T20:00:24.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.867+0000 7fa69be00640 1 --2- 172.21.15.88:0/2837785510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a40d12b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4074250 tx=0x7fa68c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b688f4fcdcbede server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.867+0000 7fa6a2000640 1 -- 172.21.15.88:0/2837785510 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c037070 con 0x7fa6a40d0e70 2024-04-26T20:00:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a2000640 1 -- 172.21.15.88:0/2837785510 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68c032440 con 0x7fa6a40d0e70 2024-04-26T20:00:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a2000640 1 -- 172.21.15.88:0/2837785510 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c032810 con 0x7fa6a40d0e70 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2837785510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 msgr2=0x7fa6a40d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2837785510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a40d12b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4074250 tx=0x7fa68c02a230 comp rx=0 tx=0).stop 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2837785510 shutdown_connections 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2837785510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a40d12b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2837785510 >> v2:172.21.15.132:3300/0 conn(0x7fa6a4146680 0x7fa6a40b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2837785510 >> 172.21.15.88:0/2837785510 conn(0x7fa6a406c740 msgr2=0x7fa6a406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2837785510 shutdown_connections 2024-04-26T20:00:24.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.868+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2837785510 wait complete. 2024-04-26T20:00:24.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.870+0000 7fa6a3e00640 1 Processor -- start 2024-04-26T20:00:24.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.870+0000 7fa6a3e00640 1 -- start start 2024-04-26T20:00:24.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a4146680 0x7fa6a41441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a4160930 0x7fa6a41f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a406e600 con 0x7fa6a40d0e70 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a406e320 con 0x7fa6a4160930 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40cd6a0 con 0x7fa6a4146680 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48326/0 (socket says 172.21.15.88:48326) 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.871+0000 7fa6a2a00640 1 -- 172.21.15.88:0/2298483440 learned_addr learned my addr 172.21.15.88:0/2298483440 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a3400640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a4160930 0x7fa6a41f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa69be00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a4146680 0x7fa6a41441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 -- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a4146680 msgr2=0x7fa6a41441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a4146680 0x7fa6a41441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 -- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a4160930 msgr2=0x7fa6a41f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a4160930 0x7fa6a41f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 -- 172.21.15.88:0/2298483440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a41f3d60 con 0x7fa6a40d0e70 2024-04-26T20:00:24.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a2a00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d800 tx=0x7fa69400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400b6c0 con 0x7fa6a40d0e70 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694019070 con 0x7fa6a40d0e70 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.872+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a4144a20 con 0x7fa6a40d0e70 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.873+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400ba80 con 0x7fa6a40d0e70 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.873+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a4138fe0 con 0x7fa6a40d0e70 2024-04-26T20:00:24.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.874+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a4144f40 con 0x7fa6a40d0e70 2024-04-26T20:00:24.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.875+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa69400bc20 con 0x7fa6a40d0e70 2024-04-26T20:00:24.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.876+0000 7fa69aa00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 0x7fa68006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:24.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.876+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa69408c040 con 0x7fa6a40d0e70 2024-04-26T20:00:24.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.876+0000 7fa69be00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 0x7fa68006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:24.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.877+0000 7fa69be00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 0x7fa68006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034000 tx=0x7fa68c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:24.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:24.881+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6a4144f40 con 0x7fa6a40d0e70 2024-04-26T20:00:25.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.083+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6a40693c0 con 0x7fa6a40d0e70 2024-04-26T20:00:25.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.085+0000 7fa69aa00640 1 -- 172.21.15.88:0/2298483440 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7fa69401c360 con 0x7fa6a40d0e70 2024-04-26T20:00:25.086 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:25.086 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 msgr2=0x7fa68006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 0x7fa68006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa68c034000 tx=0x7fa68c02e000 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 msgr2=0x7fa6a415bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d800 tx=0x7fa69400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 shutdown_connections 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6a4160930 0x7fa6a41f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa68006c8f0 0x7fa68006edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6a4146680 0x7fa6a41441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/2298483440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40d0e70 0x7fa6a415bd40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 >> 172.21.15.88:0/2298483440 conn(0x7fa6a406c740 msgr2=0x7fa6a40d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 shutdown_connections 2024-04-26T20:00:25.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.088+0000 7fa6a3e00640 1 -- 172.21.15.88:0/2298483440 wait complete. 2024-04-26T20:00:25.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:25.109 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:25.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.304+0000 7fdf80a00640 1 Processor -- start 2024-04-26T20:00:25.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.304+0000 7fdf80a00640 1 -- start start 2024-04-26T20:00:25.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf80a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdf7c0cfb90 0x7fdf7c0d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf80a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0d0170 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf80a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0d0450 con 0x7fdf7c0d2c50 2024-04-26T20:00:25.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf80a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c0d0730 con 0x7fdf7c0cfb90 2024-04-26T20:00:25.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf7a800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdf7c15f7f0 0x7fdf7c15fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55752/0 (socket says 172.21.15.88:55752) 2024-04-26T20:00:25.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf7a800640 1 -- 172.21.15.88:0/3389995289 learned_addr learned my addr 172.21.15.88:0/3389995289 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:25.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf73e00640 1 --2- 172.21.15.88:0/3389995289 >> v2:172.21.15.132:3300/0 conn(0x7fdf7c0cfb90 0x7fdf7c0d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678293770 0 0) 0x7fdf7c0d0170 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.306+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf60003520 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986191538 0 0) 0x7fdf60003520 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 >> v2:172.21.15.132:3300/0 conn(0x7fdf7c0cfb90 msgr2=0x7fdf7c0d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 --2- 172.21.15.88:0/3389995289 >> v2:172.21.15.132:3300/0 conn(0x7fdf7c0cfb90 0x7fdf7c0d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 >> v1:172.21.15.132:6789/0 conn(0x7fdf7c0d2c50 legacy=0x7fdf7c0d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0d0a10 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fdf6c001670 con 0x7fdf7c15f7f0 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 >> v1:172.21.15.88:6789/0 conn(0x7fdf7c15f7f0 legacy=0x7fdf7c15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 0x7fdf60003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf60004800 0x7fdf60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60005230 0x7fdf6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0d0730 con 0x7fdf60003a20 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6000c190 con 0x7fdf60004800 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6000c390 con 0x7fdf60005230 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.307+0000 7fdf7a800640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 0x7fdf60003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf73e00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf60004800 0x7fdf60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7b200640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60005230 0x7fdf6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7a800640 1 -- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60005230 msgr2=0x7fdf6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7a800640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60005230 0x7fdf6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7a800640 1 -- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf60004800 msgr2=0x7fdf60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7a800640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf60004800 0x7fdf60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7a800640 1 -- 172.21.15.88:0/3389995289 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60003520 con 0x7fdf60003a20 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7b200640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60005230 0x7fdf6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf73e00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf60004800 0x7fdf60004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:25.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.308+0000 7fdf7b200640 1 -- 172.21.15.88:0/3389995289 reap_dead start 2024-04-26T20:00:25.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.309+0000 7fdf7a800640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 0x7fdf60003e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c240 tx=0x7fdf60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c737e5d158f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.309+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c007d10 con 0x7fdf60003a20 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.309+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c005ad0 con 0x7fdf60003a20 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.309+0000 7fdf79e00640 1 -- 172.21.15.88:0/3389995289 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c007070 con 0x7fdf60003a20 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 -- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 msgr2=0x7fdf60003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 0x7fdf60003e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c240 tx=0x7fdf60005830 comp rx=0 tx=0).stop 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 -- 172.21.15.88:0/3389995289 shutdown_connections 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 --2- 172.21.15.88:0/3389995289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf60003a20 0x7fdf60003e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 -- 172.21.15.88:0/3389995289 >> 172.21.15.88:0/3389995289 conn(0x7fdf7c0d4f80 msgr2=0x7fdf7c0d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 -- 172.21.15.88:0/3389995289 shutdown_connections 2024-04-26T20:00:25.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.310+0000 7fdf80a00640 1 -- 172.21.15.88:0/3389995289 wait complete. 2024-04-26T20:00:25.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.312+0000 7fdf80a00640 1 Processor -- start 2024-04-26T20:00:25.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.312+0000 7fdf80a00640 1 -- start start 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60003a20 0x7fdf7c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf7c0ce4e0 0x7fdf7c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c15f5b0 con 0x7fdf7c0d3990 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c15f430 con 0x7fdf7c0ce4e0 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf80a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c15fbd0 con 0x7fdf60003a20 2024-04-26T20:00:25.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf7b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf7b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48334/0 (socket says 172.21.15.88:48334) 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.313+0000 7fdf7b200640 1 -- 172.21.15.88:0/2925938949 learned_addr learned my addr 172.21.15.88:0/2925938949 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 -- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60003a20 msgr2=0x7fdf7c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60003a20 0x7fdf7c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 -- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf7c0ce4e0 msgr2=0x7fdf7c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf7c0ce4e0 0x7fdf7c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 -- 172.21.15.88:0/2925938949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0cf310 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf7b200640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0d4f50 tx=0x7fdf7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74018070 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74003b70 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74005070 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c0cea70 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.314+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c0cf030 con 0x7fdf7c0d3990 2024-04-26T20:00:25.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.315+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c0d4350 con 0x7fdf7c0d3990 2024-04-26T20:00:25.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.317+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7401b070 con 0x7fdf7c0d3990 2024-04-26T20:00:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.318+0000 7fdf72a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.318+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf7408b060 con 0x7fdf7c0d3990 2024-04-26T20:00:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.318+0000 7fdf7a800640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.319+0000 7fdf7a800640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 0x7fdf5806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c032000 tx=0x7fdf6c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.322+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf74059520 con 0x7fdf7c0d3990 2024-04-26T20:00:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.513+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf7c0693c0 con 0x7fdf7c0d3990 2024-04-26T20:00:25.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.514+0000 7fdf72a00640 1 -- 172.21.15.88:0/2925938949 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdf74058e20 con 0x7fdf7c0d3990 2024-04-26T20:00:25.515 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 msgr2=0x7fdf5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 0x7fdf5806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c032000 tx=0x7fdf6c007000 comp rx=0 tx=0).stop 2024-04-26T20:00:25.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 msgr2=0x7fdf7c0d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0d4f50 tx=0x7fdf7400dc00 comp rx=0 tx=0).stop 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 shutdown_connections 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf7c0d3990 0x7fdf7c0d3e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf7c0ce4e0 0x7fdf7c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf5806c8f0 0x7fdf5806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 --2- 172.21.15.88:0/2925938949 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf60003a20 0x7fdf7c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.516+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 >> 172.21.15.88:0/2925938949 conn(0x7fdf7c0d4f80 msgr2=0x7fdf7c154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.517+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 shutdown_connections 2024-04-26T20:00:25.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.517+0000 7fdf80a00640 1 -- 172.21.15.88:0/2925938949 wait complete. 2024-04-26T20:00:25.534 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:25.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.715+0000 7f284b200640 1 Processor -- start 2024-04-26T20:00:25.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.715+0000 7f284b200640 1 -- start start 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f284b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f284c160d60 0x7f284c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f284b200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f284c161140 0x7f284c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f284b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f284c0b4d20 con 0x7f284c146280 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f284b200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f284c0b5000 con 0x7f284c161140 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f284b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f284c0b52e0 con 0x7f284c160d60 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.716+0000 7f2849400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f284c161140 0x7f284c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f284c161140 0x7f284c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48350/0 (socket says 172.21.15.88:48350) 2024-04-26T20:00:25.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 -- 172.21.15.88:0/4090372919 learned_addr learned my addr 172.21.15.88:0/4090372919 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 -- 172.21.15.88:0/4090372919 >> v2:172.21.15.132:3300/0 conn(0x7f284c160d60 msgr2=0x7f284c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849e00640 1 --2- 172.21.15.88:0/4090372919 >> v2:172.21.15.132:3300/0 conn(0x7f284c160d60 0x7f284c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 --2- 172.21.15.88:0/4090372919 >> v2:172.21.15.132:3300/0 conn(0x7f284c160d60 0x7f284c0b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 -- 172.21.15.88:0/4090372919 >> v1:172.21.15.132:6789/0 conn(0x7f284c146280 legacy=0x7f284c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.717+0000 7f2849400640 1 -- 172.21.15.88:0/4090372919 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c0cf410 con 0x7f284c161140 2024-04-26T20:00:25.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.718+0000 7f2849400640 1 --2- 172.21.15.88:0/4090372919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c161140 0x7f284c06dee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2838009870 tx=0x7f283802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc4c3812aecdca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.718+0000 7f2848a00640 1 -- 172.21.15.88:0/4090372919 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838037070 con 0x7f284c161140 2024-04-26T20:00:25.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.718+0000 7f2848a00640 1 -- 172.21.15.88:0/4090372919 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2838032440 con 0x7f284c161140 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.718+0000 7f2848a00640 1 -- 172.21.15.88:0/4090372919 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838032810 con 0x7f284c161140 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 -- 172.21.15.88:0/4090372919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c161140 msgr2=0x7f284c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 --2- 172.21.15.88:0/4090372919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c161140 0x7f284c06dee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2838009870 tx=0x7f283802a250 comp rx=0 tx=0).stop 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 -- 172.21.15.88:0/4090372919 shutdown_connections 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 --2- 172.21.15.88:0/4090372919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c161140 0x7f284c06dee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 --2- 172.21.15.88:0/4090372919 >> v2:172.21.15.132:3300/0 conn(0x7f284c160d60 0x7f284c0b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 -- 172.21.15.88:0/4090372919 >> 172.21.15.88:0/4090372919 conn(0x7f284c0b2900 msgr2=0x7f284c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 -- 172.21.15.88:0/4090372919 shutdown_connections 2024-04-26T20:00:25.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.719+0000 7f284b200640 1 -- 172.21.15.88:0/4090372919 wait complete. 2024-04-26T20:00:25.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.721+0000 7f284b200640 1 Processor -- start 2024-04-26T20:00:25.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.721+0000 7f284b200640 1 -- start start 2024-04-26T20:00:25.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f284c160d60 0x7f284c1f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f284c161140 0x7f284c13bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f284c06e420 con 0x7f284c146280 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0b6070 con 0x7f284c160d60 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f284b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f284c06e5a0 con 0x7f284c161140 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48352/0 (socket says 172.21.15.88:48352) 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 -- 172.21.15.88:0/3906637898 learned_addr learned my addr 172.21.15.88:0/3906637898 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 -- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f284c161140 msgr2=0x7f284c13bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f284c161140 0x7f284c13bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 -- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f284c160d60 msgr2=0x7f284c1f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849400640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f284c160d60 0x7f284c1f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f284c160d60 0x7f284c1f8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.722+0000 7f2849e00640 1 -- 172.21.15.88:0/3906637898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1f82c0 con 0x7f284c146280 2024-04-26T20:00:25.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f2849e00640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c7f0 tx=0x7f283c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c015070 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f283c004240 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c007620 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c13c270 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.723+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1f8810 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.724+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0693c0 con 0x7f284c146280 2024-04-26T20:00:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.725+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f283c0079d0 con 0x7f284c146280 2024-04-26T20:00:25.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.726+0000 7f2842000640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 0x7f282c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:25.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.726+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f283c08bbd0 con 0x7f284c146280 2024-04-26T20:00:25.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.726+0000 7f2849400640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 0x7f282c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:25.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.727+0000 7f2849400640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 0x7f282c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2838034000 tx=0x7f283802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:25.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.730+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f283c059ff0 con 0x7f284c146280 2024-04-26T20:00:25.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.935+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f284c159cb0 con 0x7f284c146280 2024-04-26T20:00:25.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.936+0000 7f2842000640 1 -- 172.21.15.88:0/3906637898 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f283c0598f0 con 0x7f284c146280 2024-04-26T20:00:25.936 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 msgr2=0x7f282c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 0x7f282c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2838034000 tx=0x7f283802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 msgr2=0x7f284c13b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f283c00c7f0 tx=0x7f283c00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 shutdown_connections 2024-04-26T20:00:25.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f282c06c8f0 0x7f282c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.938+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f284c161140 0x7f284c13bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.939+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f284c160d60 0x7f284c1f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.939+0000 7f284b200640 1 --2- 172.21.15.88:0/3906637898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f284c146280 0x7f284c13b1d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.939+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 >> 172.21.15.88:0/3906637898 conn(0x7f284c0b2900 msgr2=0x7f284c1457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.939+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 shutdown_connections 2024-04-26T20:00:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:25.939+0000 7f284b200640 1 -- 172.21.15.88:0/3906637898 wait complete. 2024-04-26T20:00:25.957 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:26.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.155+0000 7f9d98e00640 1 Processor -- start 2024-04-26T20:00:26.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.155+0000 7f9d98e00640 1 -- start start 2024-04-26T20:00:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.156+0000 7f9d98e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9d940b6f40 0x7f9d940b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d98e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d9406c350 0x7f9d9406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d98e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d940b6430 con 0x7f9d9415f220 2024-04-26T20:00:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d98e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9406ccc0 con 0x7f9d940b6f40 2024-04-26T20:00:26.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d98e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f9d940b12c0 con 0x7f9d9406c350 2024-04-26T20:00:26.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d8be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9d940b6f40 0x7f9d940b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d8be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9d940b6f40 0x7f9d940b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48354/0 (socket says 172.21.15.88:48354) 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d92800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9d9406c350 0x7f9d9406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.157+0000 7f9d8be00640 1 -- 172.21.15.88:0/3917935458 learned_addr learned my addr 172.21.15.88:0/3917935458 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.158+0000 7f9d8be00640 1 -- 172.21.15.88:0/3917935458 >> v2:172.21.15.132:3300/0 conn(0x7f9d9406c350 msgr2=0x7f9d9406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.158+0000 7f9d8be00640 1 --2- 172.21.15.88:0/3917935458 >> v2:172.21.15.132:3300/0 conn(0x7f9d9406c350 0x7f9d9406c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.158+0000 7f9d8be00640 1 -- 172.21.15.88:0/3917935458 >> v1:172.21.15.132:6789/0 conn(0x7f9d9415f220 legacy=0x7f9d940b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.158+0000 7f9d8be00640 1 -- 172.21.15.88:0/3917935458 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d940b15a0 con 0x7f9d940b6f40 2024-04-26T20:00:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.159+0000 7f9d8be00640 1 --2- 172.21.15.88:0/3917935458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d940b6f40 0x7f9d940b6860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d940b53b0 tx=0x7f9d7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f98227efbce384 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.159+0000 7f9d91400640 1 -- 172.21.15.88:0/3917935458 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c037070 con 0x7f9d940b6f40 2024-04-26T20:00:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.159+0000 7f9d91400640 1 -- 172.21.15.88:0/3917935458 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c032440 con 0x7f9d940b6f40 2024-04-26T20:00:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.159+0000 7f9d91400640 1 -- 172.21.15.88:0/3917935458 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c032810 con 0x7f9d940b6f40 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 -- 172.21.15.88:0/3917935458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d940b6f40 msgr2=0x7f9d940b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 --2- 172.21.15.88:0/3917935458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d940b6f40 0x7f9d940b6860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d940b53b0 tx=0x7f9d7c02a270 comp rx=0 tx=0).stop 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 -- 172.21.15.88:0/3917935458 shutdown_connections 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 --2- 172.21.15.88:0/3917935458 >> v2:172.21.15.132:3300/0 conn(0x7f9d9406c350 0x7f9d9406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 --2- 172.21.15.88:0/3917935458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d940b6f40 0x7f9d940b6860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 -- 172.21.15.88:0/3917935458 >> 172.21.15.88:0/3917935458 conn(0x7f9d940b4420 msgr2=0x7f9d940d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 -- 172.21.15.88:0/3917935458 shutdown_connections 2024-04-26T20:00:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.160+0000 7f9d98e00640 1 -- 172.21.15.88:0/3917935458 wait complete. 2024-04-26T20:00:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.162+0000 7f9d98e00640 1 Processor -- start 2024-04-26T20:00:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.162+0000 7f9d98e00640 1 -- start start 2024-04-26T20:00:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.162+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9406c350 0x7f9d94145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.162+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 0x7f9d9415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d98e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d98e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9415f600 con 0x7f9d9415f220 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d98e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9d940b5800 con 0x7f9d940b6f40 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d98e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9d940b0f70 con 0x7f9d9406c350 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48362/0 (socket says 172.21.15.88:48362) 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 -- 172.21.15.88:0/4277250384 learned_addr learned my addr 172.21.15.88:0/4277250384 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d8be00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 0x7f9d9415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d91e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9406c350 0x7f9d94145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 -- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9406c350 msgr2=0x7f9d94145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9406c350 0x7f9d94145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 -- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 msgr2=0x7f9d9415f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 0x7f9d9415f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d92800640 1 -- 172.21.15.88:0/4277250384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d94157330 con 0x7f9d9415f220 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.163+0000 7f9d8be00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 0x7f9d9415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d92800640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d30 tx=0x7f9d8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c018020 con 0x7f9d9415f220 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8c014480 con 0x7f9d9415f220 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d94157610 con 0x7f9d9415f220 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c014870 con 0x7f9d9415f220 2024-04-26T20:00:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.164+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9415fcc0 con 0x7f9d9415f220 2024-04-26T20:00:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.165+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d94069650 con 0x7f9d9415f220 2024-04-26T20:00:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.166+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d8c01b020 con 0x7f9d9415f220 2024-04-26T20:00:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.167+0000 7f9d8a000640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.167+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d8c08c640 con 0x7f9d9415f220 2024-04-26T20:00:26.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.167+0000 7f9d91e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.168+0000 7f9d91e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d840015a0 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.171+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d8c022030 con 0x7f9d9415f220 2024-04-26T20:00:26.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.377+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d940afb70 con 0x7f9d9415f220 2024-04-26T20:00:26.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.378+0000 7f9d8a000640 1 -- 172.21.15.88:0/4277250384 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9d8c010040 con 0x7f9d9415f220 2024-04-26T20:00:26.378 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 msgr2=0x7f9d7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 0x7f9d7006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d840015a0 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 msgr2=0x7f9d94146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d30 tx=0x7f9d8c00c490 comp rx=0 tx=0).stop 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 shutdown_connections 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d9415f220 0x7f9d94146040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d940b6f40 0x7f9d9415f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9d7006c8f0 0x7f9d7006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 --2- 172.21.15.88:0/4277250384 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9d9406c350 0x7f9d94145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.380+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 >> 172.21.15.88:0/4277250384 conn(0x7f9d940b4420 msgr2=0x7f9d94068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.381+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 shutdown_connections 2024-04-26T20:00:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.381+0000 7f9d98e00640 1 -- 172.21.15.88:0/4277250384 wait complete. 2024-04-26T20:00:26.458 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-04-26T20:00:26.458 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:00:26.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:26.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.663+0000 7f65a3200640 1 Processor -- start 2024-04-26T20:00:26.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.663+0000 7f65a3200640 1 -- start start 2024-04-26T20:00:26.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a3200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f65a40d0e70 0x7f65a40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a3200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a4146a60 con 0x7f65a4146680 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a406dd60 con 0x7f65a40b3b20 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a3200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f65a406e970 con 0x7f65a40d0e70 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f65a40d0e70 0x7f65a40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f65a4146680 0x7f65a40b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55760/0 (socket says 172.21.15.88:55760) 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1e00640 1 -- 172.21.15.88:0/768190945 learned_addr learned my addr 172.21.15.88:0/768190945 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:26.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1400640 1 --2- 172.21.15.88:0/768190945 >> v2:172.21.15.88:3300/0 conn(0x7f65a40d0e70 0x7f65a40d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48376/0 (socket says 172.21.15.88:48376) 2024-04-26T20:00:26.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1400640 1 -- 172.21.15.88:0/768190945 >> v1:172.21.15.132:6789/0 conn(0x7f65a40b3b20 legacy=0x7f65a40b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:26.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1400640 1 -- 172.21.15.88:0/768190945 >> v1:172.21.15.88:6789/0 conn(0x7f65a4146680 legacy=0x7f65a40b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.665+0000 7f65a1400640 1 -- 172.21.15.88:0/768190945 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a406e040 con 0x7f65a40d0e70 2024-04-26T20:00:26.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.666+0000 7f65a1400640 1 --2- 172.21.15.88:0/768190945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40d12b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65900097a0 tx=0x7f659002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769807a213dbdcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.667+0000 7f65a0a00640 1 -- 172.21.15.88:0/768190945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590037070 con 0x7f65a40d0e70 2024-04-26T20:00:26.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.667+0000 7f65a0a00640 1 -- 172.21.15.88:0/768190945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590032440 con 0x7f65a40d0e70 2024-04-26T20:00:26.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.667+0000 7f65a0a00640 1 -- 172.21.15.88:0/768190945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590032810 con 0x7f65a40d0e70 2024-04-26T20:00:26.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.667+0000 7f65a3200640 1 -- 172.21.15.88:0/768190945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 msgr2=0x7f65a40d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.667+0000 7f65a3200640 1 --2- 172.21.15.88:0/768190945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40d12b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f65900097a0 tx=0x7f659002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.668+0000 7f65a3200640 1 -- 172.21.15.88:0/768190945 shutdown_connections 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.668+0000 7f65a3200640 1 --2- 172.21.15.88:0/768190945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40d12b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.668+0000 7f65a3200640 1 -- 172.21.15.88:0/768190945 >> 172.21.15.88:0/768190945 conn(0x7f65a406c740 msgr2=0x7f65a406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.668+0000 7f65a3200640 1 -- 172.21.15.88:0/768190945 shutdown_connections 2024-04-26T20:00:26.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.668+0000 7f65a3200640 1 -- 172.21.15.88:0/768190945 wait complete. 2024-04-26T20:00:26.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.669+0000 7f65a3200640 1 Processor -- start 2024-04-26T20:00:26.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.670+0000 7f65a3200640 1 -- start start 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.670+0000 7f65a3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 0x7f65a415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.670+0000 7f65a3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.670+0000 7f65a3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 0x7f65a415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.670+0000 7f65a3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40d17f0 con 0x7f65a40d0e70 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f65a40b60a0 con 0x7f65a40b3b20 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f65a406e600 con 0x7f65a4146680 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48382/0 (socket says 172.21.15.88:48382) 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 -- 172.21.15.88:0/1462786106 learned_addr learned my addr 172.21.15.88:0/1462786106 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1e00640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 0x7f65a415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a2800640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 0x7f65a415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 -- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 msgr2=0x7f65a415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 0x7f65a415d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 -- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 msgr2=0x7f65a415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 0x7f65a415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1400640 1 -- 172.21.15.88:0/1462786106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a40caac0 con 0x7f65a40d0e70 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a1e00640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 0x7f65a415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.671+0000 7f65a2800640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 0x7f65a415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f65a1400640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f659002a220 tx=0x7f65900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590041070 con 0x7f65a40d0e70 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659003c890 con 0x7f65a40d0e70 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590037040 con 0x7f65a40d0e70 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a40cada0 con 0x7f65a40d0e70 2024-04-26T20:00:26.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.672+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a40cb360 con 0x7f65a40d0e70 2024-04-26T20:00:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.673+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a40693c0 con 0x7f65a40d0e70 2024-04-26T20:00:26.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.674+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6590044070 con 0x7f65a40d0e70 2024-04-26T20:00:26.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.675+0000 7f659a000640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:26.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.675+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65900ac6c0 con 0x7f65a40d0e70 2024-04-26T20:00:26.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.676+0000 7f65a1e00640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:26.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.676+0000 7f65a1e00640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 0x7f658406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6594000f60 tx=0x7f6594009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:26.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.680+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f659007ab80 con 0x7f65a40d0e70 2024-04-26T20:00:26.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.881+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65a415eff0 con 0x7f65a40d0e70 2024-04-26T20:00:26.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.882+0000 7f659a000640 1 -- 172.21.15.88:0/1462786106 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f659007a480 con 0x7f65a40d0e70 2024-04-26T20:00:26.883 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:26.884 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T19:59:47.277716+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 msgr2=0x7f658406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 0x7f658406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6594000f60 tx=0x7f6594009040 comp rx=0 tx=0).stop 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 msgr2=0x7f65a40ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f659002a220 tx=0x7f65900046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 shutdown_connections 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65a4146680 0x7f65a415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65a40d0e70 0x7f65a40ca0b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 --2- 172.21.15.88:0/1462786106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65a40b3b20 0x7f65a415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:26.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 >> 172.21.15.88:0/1462786106 conn(0x7f65a406c740 msgr2=0x7f65a40cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:26.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 shutdown_connections 2024-04-26T20:00:26.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:26.885+0000 7f65a3200640 1 -- 172.21.15.88:0/1462786106 wait complete. 2024-04-26T20:00:26.904 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:26.905 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:27.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.097+0000 7fdf8c400640 1 Processor -- start 2024-04-26T20:00:27.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.097+0000 7fdf8c400640 1 -- start start 2024-04-26T20:00:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 0x7fdf840b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8c400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdf84073da0 0x7fdf8406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8c400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf840afed0 con 0x7fdf8406ccb0 2024-04-26T20:00:27.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8c400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8406d0e0 con 0x7fdf84073da0 2024-04-26T20:00:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdf840ade20 con 0x7fdf84147f20 2024-04-26T20:00:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8a400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdf8406ccb0 0x7fdf84072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55774/0 (socket says 172.21.15.88:55774) 2024-04-26T20:00:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.099+0000 7fdf8a400640 1 -- 172.21.15.88:0/1756803438 learned_addr learned my addr 172.21.15.88:0/1756803438 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf89a00640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 0x7fdf840b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663791123 0 0) 0x7fdf7c001490 con 0x7fdf8406ccb0 2024-04-26T20:00:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf89000640 1 --2- 172.21.15.88:0/1756803438 >> v2:172.21.15.132:3300/0 conn(0x7fdf84073da0 0x7fdf8406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf6c003520 con 0x7fdf8406ccb0 2024-04-26T20:00:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2244588558 0 0) 0x7fdf6c003520 con 0x7fdf8406ccb0 2024-04-26T20:00:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 msgr2=0x7fdf840b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 0x7fdf840b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 >> v2:172.21.15.132:3300/0 conn(0x7fdf84073da0 msgr2=0x7fdf8406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 --2- 172.21.15.88:0/1756803438 >> v2:172.21.15.132:3300/0 conn(0x7fdf84073da0 0x7fdf8406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf840ae100 con 0x7fdf8406ccb0 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fdf7c001bc0 con 0x7fdf8406ccb0 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 >> v1:172.21.15.88:6789/0 conn(0x7fdf8406ccb0 legacy=0x7fdf84072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0039b0 0x7fdf6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.100+0000 7fdf83e00640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf83e00640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 0x7fdf6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8406d0e0 con 0x7fdf6c004800 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840ade20 con 0x7fdf6c0039b0 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c00b9c0 con 0x7fdf84147f20 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 -- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 msgr2=0x7fdf6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf84147f20 0x7fdf6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 -- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0039b0 msgr2=0x7fdf6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0039b0 0x7fdf6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.101+0000 7fdf89000640 1 -- 172.21.15.88:0/1756803438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c003520 con 0x7fdf6c004800 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.102+0000 7fdf89000640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009ac0 tx=0x7fdf6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0337b4ce6fd1f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.102+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70036070 con 0x7fdf6c004800 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.102+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70031400 con 0x7fdf6c004800 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.102+0000 7fdf83e00640 1 -- 172.21.15.88:0/1756803438 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009690 con 0x7fdf6c004800 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 -- 172.21.15.88:0/1756803438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c004800 msgr2=0x7fdf6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 --2- 172.21.15.88:0/1756803438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009ac0 tx=0x7fdf6c005830 comp rx=0 tx=0).stop 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8a400640 1 -- 172.21.15.88:0/1756803438 reap_dead start 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 -- 172.21.15.88:0/1756803438 shutdown_connections 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 -- 172.21.15.88:0/1756803438 >> 172.21.15.88:0/1756803438 conn(0x7fdf840b2d50 msgr2=0x7fdf840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 -- 172.21.15.88:0/1756803438 shutdown_connections 2024-04-26T20:00:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.103+0000 7fdf8c400640 1 -- 172.21.15.88:0/1756803438 wait complete. 2024-04-26T20:00:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.105+0000 7fdf8c400640 1 Processor -- start 2024-04-26T20:00:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.105+0000 7fdf8c400640 1 -- start start 2024-04-26T20:00:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf6c004800 0x7fdf84162460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf84142a60 0x7fdf841403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840ae9a0 con 0x7fdf841408e0 2024-04-26T20:00:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdf840ae6c0 con 0x7fdf84142a60 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.106+0000 7fdf8c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdf84074200 con 0x7fdf6c004800 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48402/0 (socket says 172.21.15.88:48402) 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 -- 172.21.15.88:0/2774809661 learned_addr learned my addr 172.21.15.88:0/2774809661 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf89a00640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf6c004800 0x7fdf84162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf89000640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf84142a60 0x7fdf841403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 -- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf6c004800 msgr2=0x7fdf84162460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf6c004800 0x7fdf84162460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 -- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf84142a60 msgr2=0x7fdf841403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf84142a60 0x7fdf841403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 -- 172.21.15.88:0/2774809661 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf84142f40 con 0x7fdf841408e0 2024-04-26T20:00:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.107+0000 7fdf8a400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0012e0 tx=0x7fdf7c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.110+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00a610 con 0x7fdf841408e0 2024-04-26T20:00:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.110+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c00a7f0 con 0x7fdf841408e0 2024-04-26T20:00:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.110+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf84143220 con 0x7fdf841408e0 2024-04-26T20:00:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.110+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00abe0 con 0x7fdf841408e0 2024-04-26T20:00:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.110+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841f6100 con 0x7fdf841408e0 2024-04-26T20:00:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.112+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf84069650 con 0x7fdf841408e0 2024-04-26T20:00:27.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.112+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf7c011070 con 0x7fdf841408e0 2024-04-26T20:00:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.113+0000 7fdf81600640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 0x7fdf6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.114+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdf7c08b630 con 0x7fdf841408e0 2024-04-26T20:00:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.114+0000 7fdf89a00640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 0x7fdf6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.115+0000 7fdf89a00640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 0x7fdf6406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf78004770 tx=0x7fdf78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.119+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf7c01a060 con 0x7fdf841408e0 2024-04-26T20:00:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.318+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf840b1600 con 0x7fdf841408e0 2024-04-26T20:00:27.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.319+0000 7fdf81600640 1 -- 172.21.15.88:0/2774809661 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fdf840afed0 con 0x7fdf841408e0 2024-04-26T20:00:27.319 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 msgr2=0x7fdf6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 0x7fdf6406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf78004770 tx=0x7fdf78008040 comp rx=0 tx=0).stop 2024-04-26T20:00:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 msgr2=0x7fdf841f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0012e0 tx=0x7fdf7c00cf00 comp rx=0 tx=0).stop 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 shutdown_connections 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf841408e0 0x7fdf841f4f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf84142a60 0x7fdf841403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdf6406c8f0 0x7fdf6406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 --2- 172.21.15.88:0/2774809661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdf6c004800 0x7fdf84162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.321+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 >> 172.21.15.88:0/2774809661 conn(0x7fdf840b2d50 msgr2=0x7fdf840b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.322+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 shutdown_connections 2024-04-26T20:00:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.322+0000 7fdf8c400640 1 -- 172.21.15.88:0/2774809661 wait complete. 2024-04-26T20:00:27.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.524+0000 7f74d3e00640 1 Processor -- start 2024-04-26T20:00:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.524+0000 7f74d3e00640 1 -- start start 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 0x7f74d40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f74d4146280 0x7f74d41466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d40b4d20 con 0x7f74d4161140 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f74d406e5f0 con 0x7f74d4146280 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f74d4146c30 con 0x7f74d4160d60 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74cbe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f74d4161140 0x7f74d406dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55788/0 (socket says 172.21.15.88:55788) 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74cbe00640 1 -- 172.21.15.88:0/3175770851 learned_addr learned my addr 172.21.15.88:0/3175770851 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.525+0000 7f74d3400640 1 --2- 172.21.15.88:0/3175770851 >> v2:172.21.15.88:3300/0 conn(0x7f74d4146280 0x7f74d41466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d2000640 1 -- 172.21.15.88:0/3175770851 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165743328 0 0) 0x7f74d40b4d20 con 0x7f74d4161140 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d2000640 1 -- 172.21.15.88:0/3175770851 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74b8003520 con 0x7f74d4161140 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d2a00640 1 --2- 172.21.15.88:0/3175770851 >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 0x7f74d40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d3400640 1 -- 172.21.15.88:0/3175770851 >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 msgr2=0x7f74d40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d3400640 1 --2- 172.21.15.88:0/3175770851 >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 0x7f74d40b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d3400640 1 -- 172.21.15.88:0/3175770851 >> v1:172.21.15.88:6789/0 conn(0x7f74d4161140 legacy=0x7f74d406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d3400640 1 -- 172.21.15.88:0/3175770851 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d40b5000 con 0x7f74d4146280 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.526+0000 7f74d2a00640 1 --2- 172.21.15.88:0/3175770851 >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 0x7f74d40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.527+0000 7f74d3400640 1 --2- 172.21.15.88:0/3175770851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41466f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009d90 tx=0x7f74cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7f8dc92e6dd80d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.527+0000 7f74d2000640 1 -- 172.21.15.88:0/3175770851 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc007740 con 0x7f74d4146280 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.527+0000 7f74d2000640 1 -- 172.21.15.88:0/3175770851 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc0078e0 con 0x7f74d4146280 2024-04-26T20:00:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.527+0000 7f74d2000640 1 -- 172.21.15.88:0/3175770851 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc007cb0 con 0x7f74d4146280 2024-04-26T20:00:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 -- 172.21.15.88:0/3175770851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 msgr2=0x7f74d41466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 --2- 172.21.15.88:0/3175770851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41466f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009d90 tx=0x7f74cc02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 -- 172.21.15.88:0/3175770851 shutdown_connections 2024-04-26T20:00:27.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 --2- 172.21.15.88:0/3175770851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41466f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 --2- 172.21.15.88:0/3175770851 >> v2:172.21.15.132:3300/0 conn(0x7f74d4160d60 0x7f74d40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 -- 172.21.15.88:0/3175770851 >> 172.21.15.88:0/3175770851 conn(0x7f74d40b2900 msgr2=0x7f74d40b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 -- 172.21.15.88:0/3175770851 shutdown_connections 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.528+0000 7f74d3e00640 1 -- 172.21.15.88:0/3175770851 wait complete. 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.529+0000 7f74d3e00640 1 Processor -- start 2024-04-26T20:00:27.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 -- start start 2024-04-26T20:00:27.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74d4160d60 0x7f74d40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74d4161140 0x7f74d40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74d40cf6f0 con 0x7f74d4146280 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f74d40cf410 con 0x7f74d4161140 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.530+0000 7f74d3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f74d40b6070 con 0x7f74d4160d60 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48426/0 (socket says 172.21.15.88:48426) 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 -- 172.21.15.88:0/1617849823 learned_addr learned my addr 172.21.15.88:0/1617849823 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 -- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74d4160d60 msgr2=0x7f74d40cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74d4160d60 0x7f74d40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 -- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74d4161140 msgr2=0x7f74d40ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74d4161140 0x7f74d40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 -- 172.21.15.88:0/1617849823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d415d130 con 0x7f74d4146280 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.531+0000 7f74d2a00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c7f0 tx=0x7f74c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.532+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4015070 con 0x7f74d4146280 2024-04-26T20:00:27.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.532+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4007a30 con 0x7f74d4146280 2024-04-26T20:00:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.532+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d415d410 con 0x7f74d4146280 2024-04-26T20:00:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.532+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4007e00 con 0x7f74d4146280 2024-04-26T20:00:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.532+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d415d9d0 con 0x7f74d4146280 2024-04-26T20:00:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.533+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d4161770 con 0x7f74d4146280 2024-04-26T20:00:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.534+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74c4018070 con 0x7f74d4146280 2024-04-26T20:00:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.535+0000 7f74caa00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.535+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5854+0+0 (secure 0 0 0) 0x7f74c408bb60 con 0x7f74d4146280 2024-04-26T20:00:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.535+0000 7f74cbe00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.536+0000 7f74cbe00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000f60 tx=0x7f74bc0078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.540+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74d4161770 con 0x7f74d4146280 2024-04-26T20:00:27.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.743+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f74d40693c0 con 0x7f74d4146280 2024-04-26T20:00:27.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.744+0000 7f74caa00640 1 -- 172.21.15.88:0/1617849823 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f74c4059f90 con 0x7f74d4146280 2024-04-26T20:00:27.745 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 msgr2=0x7f74b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 0x7f74b006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000f60 tx=0x7f74bc0078f0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 msgr2=0x7f74d41f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c7f0 tx=0x7f74c400cbf0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 shutdown_connections 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f74d4161140 0x7f74d40ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f74b006c8f0 0x7f74b006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f74d4160d60 0x7f74d40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 --2- 172.21.15.88:0/1617849823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74d4146280 0x7f74d41f42d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 >> 172.21.15.88:0/1617849823 conn(0x7f74d40b2900 msgr2=0x7f74d406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 shutdown_connections 2024-04-26T20:00:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.747+0000 7f74d3e00640 1 -- 172.21.15.88:0/1617849823 wait complete. 2024-04-26T20:00:27.766 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:27.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.972+0000 7f2e7ac00640 1 Processor -- start 2024-04-26T20:00:27.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.972+0000 7f2e7ac00640 1 -- start start 2024-04-26T20:00:27.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.973+0000 7f2e7ac00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2e7406ce80 0x7f2e740b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.973+0000 7f2e7ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e740b5ca0 0x7f2e740b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.973+0000 7f2e7ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e740af800 con 0x7f2e7415d100 2024-04-26T20:00:27.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.973+0000 7f2e7ac00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f2e740ae340 con 0x7f2e7406ce80 2024-04-26T20:00:27.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.973+0000 7f2e7ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2e740afaa0 con 0x7f2e740b5ca0 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2e7406ce80 0x7f2e740b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2e7406ce80 0x7f2e740b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48442/0 (socket says 172.21.15.88:48442) 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 -- 172.21.15.88:0/2565185429 learned_addr learned my addr 172.21.15.88:0/2565185429 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e78c00640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e740b5ca0 0x7f2e740b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 -- 172.21.15.88:0/2565185429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e740b5ca0 msgr2=0x7f2e740b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e740b5ca0 0x7f2e740b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 -- 172.21.15.88:0/2565185429 >> v1:172.21.15.132:6789/0 conn(0x7f2e7415d100 legacy=0x7f2e7406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.974+0000 7f2e6b400640 1 -- 172.21.15.88:0/2565185429 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e740afd80 con 0x7f2e7406ce80 2024-04-26T20:00:27.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.975+0000 7f2e6b400640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e7406ce80 0x7f2e740b3c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e66a9c492a53cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e73400640 1 -- 172.21.15.88:0/2565185429 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037070 con 0x7f2e7406ce80 2024-04-26T20:00:27.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e73400640 1 -- 172.21.15.88:0/2565185429 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c032440 con 0x7f2e7406ce80 2024-04-26T20:00:27.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e73400640 1 -- 172.21.15.88:0/2565185429 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c032810 con 0x7f2e7406ce80 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 -- 172.21.15.88:0/2565185429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e7406ce80 msgr2=0x7f2e740b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e7406ce80 0x7f2e740b3c30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02a270 comp rx=0 tx=0).stop 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 -- 172.21.15.88:0/2565185429 shutdown_connections 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e740b5ca0 0x7f2e740b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/2565185429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e7406ce80 0x7f2e740b3c30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.976+0000 7f2e7ac00640 1 -- 172.21.15.88:0/2565185429 >> 172.21.15.88:0/2565185429 conn(0x7f2e740b2d50 msgr2=0x7f2e740b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:27.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.977+0000 7f2e7ac00640 1 -- 172.21.15.88:0/2565185429 shutdown_connections 2024-04-26T20:00:27.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.977+0000 7f2e7ac00640 1 -- 172.21.15.88:0/2565185429 wait complete. 2024-04-26T20:00:27.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.978+0000 7f2e7ac00640 1 Processor -- start 2024-04-26T20:00:27.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.979+0000 7f2e7ac00640 1 -- start start 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.979+0000 7f2e7ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 0x7f2e741f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.979+0000 7f2e7ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.979+0000 7f2e7ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7415d100 0x7f2e7415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.979+0000 7f2e7ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7409ee50 con 0x7f2e740b5ca0 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e7ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2e740b0340 con 0x7f2e7415d100 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e7ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2e74160b70 con 0x7f2e7406ce80 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48450/0 (socket says 172.21.15.88:48450) 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 -- 172.21.15.88:0/1030500904 learned_addr learned my addr 172.21.15.88:0/1030500904 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e73e00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 0x7f2e741f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e78c00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7415d100 0x7f2e7415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 -- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 msgr2=0x7f2e741f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 0x7f2e741f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 -- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7415d100 msgr2=0x7f2e7415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7415d100 0x7f2e7415b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e6b400640 1 -- 172.21.15.88:0/1030500904 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e741f5d30 con 0x7f2e740b5ca0 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.980+0000 7f2e73e00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 0x7f2e741f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:27.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e6b400640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c037040 con 0x7f2e740b5ca0 2024-04-26T20:00:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c042070 con 0x7f2e740b5ca0 2024-04-26T20:00:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7409ba50 con 0x7f2e740b5ca0 2024-04-26T20:00:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c030900 con 0x7f2e740b5ca0 2024-04-26T20:00:27.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.981+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7409bfd0 con 0x7f2e740b5ca0 2024-04-26T20:00:27.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.982+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e74069650 con 0x7f2e740b5ca0 2024-04-26T20:00:27.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.983+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e5c030aa0 con 0x7f2e740b5ca0 2024-04-26T20:00:27.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.984+0000 7f2e70c00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.984+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5854+0+0 (secure 0 0 0) 0x7f2e5c0ac550 con 0x7f2e740b5ca0 2024-04-26T20:00:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.984+0000 7f2e73e00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:27.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.985+0000 7f2e73e00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 0x7f2e5006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e640012e0 tx=0x7f2e64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:27.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:27.991+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e741f5d30 con 0x7f2e740b5ca0 2024-04-26T20:00:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.199+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2e740b0e20 con 0x7f2e740b5ca0 2024-04-26T20:00:28.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.200+0000 7f2e70c00640 1 -- 172.21.15.88:0/1030500904 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f2e5c07a860 con 0x7f2e740b5ca0 2024-04-26T20:00:28.201 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 msgr2=0x7f2e5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 0x7f2e5006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e640012e0 tx=0x7f2e64009040 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 msgr2=0x7f2e7414d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c000c00 tx=0x7f2e5c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 shutdown_connections 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e7415d100 0x7f2e7415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e740b5ca0 0x7f2e7414d7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e5006c8f0 0x7f2e5006edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 --2- 172.21.15.88:0/1030500904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2e7406ce80 0x7f2e741f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 >> 172.21.15.88:0/1030500904 conn(0x7f2e740b2d50 msgr2=0x7f2e7406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 shutdown_connections 2024-04-26T20:00:28.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.203+0000 7f2e7ac00640 1 -- 172.21.15.88:0/1030500904 wait complete. 2024-04-26T20:00:28.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-04-26T20:00:28.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.418+0000 7efedbe00640 1 Processor -- start 2024-04-26T20:00:28.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.418+0000 7efedbe00640 1 -- start start 2024-04-26T20:00:28.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efedc0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efedc0b5d40 0x7efedc0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedbe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efedc0d1790 con 0x7efedc0b5660 2024-04-26T20:00:28.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedbe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7efedc0727f0 con 0x7efedc0b5d40 2024-04-26T20:00:28.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efedc0b4a10 con 0x7efedc0d0e70 2024-04-26T20:00:28.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efeda000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7efedc0b5660 0x7efedc0ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55792/0 (socket says 172.21.15.88:55792) 2024-04-26T20:00:28.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efeda000640 1 -- 172.21.15.88:0/3457308159 learned_addr learned my addr 172.21.15.88:0/3457308159 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedaa00640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efedc0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efedb400640 1 --2- 172.21.15.88:0/3457308159 >> v2:172.21.15.132:3300/0 conn(0x7efedc0b5d40 0x7efedc0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.420+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177248861 0 0) 0x7efedc0d1790 con 0x7efedc0b5660 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec0003520 con 0x7efedc0b5660 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2625111777 0 0) 0x7efec0003520 con 0x7efedc0b5660 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 msgr2=0x7efedc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efedc0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 >> v2:172.21.15.132:3300/0 conn(0x7efedc0b5d40 msgr2=0x7efedc0aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 --2- 172.21.15.88:0/3457308159 >> v2:172.21.15.132:3300/0 conn(0x7efedc0b5d40 0x7efedc0aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc0b4cf0 con 0x7efedc0b5660 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7efec8001670 con 0x7efedc0b5660 2024-04-26T20:00:28.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 >> v1:172.21.15.88:6789/0 conn(0x7efedc0b5660 legacy=0x7efedc0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efec00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0003f30 0x7efec0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efec0004d40 0x7efec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0727f0 con 0x7efec0004d40 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0b4a10 con 0x7efec0003f30 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efec000ba10 con 0x7efedc0d0e70 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.421+0000 7efedb400640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efec0004d40 0x7efec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedaa00640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efec00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efeda000640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0003f30 0x7efec0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedb400640 1 -- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 msgr2=0x7efec00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedb400640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc0d0e70 0x7efec00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedb400640 1 -- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0003f30 msgr2=0x7efec0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedb400640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0003f30 0x7efec0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efedb400640 1 -- 172.21.15.88:0/3457308159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec0003520 con 0x7efec0004d40 2024-04-26T20:00:28.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.422+0000 7efeda000640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0003f30 0x7efec0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.423+0000 7efedb400640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efec0004d40 0x7efec00051c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efed400e170 tx=0x7efec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb15c7e219be664 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.423+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4034020 con 0x7efec0004d40 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.423+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4030430 con 0x7efec0004d40 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.423+0000 7efed9600640 1 -- 172.21.15.88:0/3457308159 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4009c10 con 0x7efec0004d40 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 -- 172.21.15.88:0/3457308159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efec0004d40 msgr2=0x7efec00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 --2- 172.21.15.88:0/3457308159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efec0004d40 0x7efec00051c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efed400e170 tx=0x7efec0005830 comp rx=0 tx=0).stop 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedb400640 1 -- 172.21.15.88:0/3457308159 reap_dead start 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 -- 172.21.15.88:0/3457308159 shutdown_connections 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 -- 172.21.15.88:0/3457308159 >> 172.21.15.88:0/3457308159 conn(0x7efedc06c740 msgr2=0x7efedc06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 -- 172.21.15.88:0/3457308159 shutdown_connections 2024-04-26T20:00:28.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.424+0000 7efedbe00640 1 -- 172.21.15.88:0/3457308159 wait complete. 2024-04-26T20:00:28.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.426+0000 7efedbe00640 1 Processor -- start 2024-04-26T20:00:28.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.426+0000 7efedbe00640 1 -- start start 2024-04-26T20:00:28.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0004d40 0x7efedc1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc142bf0 0x7efedc143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0b05b0 con 0x7efedc15f4d0 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0b0430 con 0x7efec0004d40 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efedc15f1a0 con 0x7efedc142bf0 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48460/0 (socket says 172.21.15.88:48460) 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 -- 172.21.15.88:0/631878532 learned_addr learned my addr 172.21.15.88:0/631878532 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efedaa00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0004d40 0x7efedc1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 -- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc142bf0 msgr2=0x7efedc143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc142bf0 0x7efedc143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 -- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0004d40 msgr2=0x7efedc1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0004d40 0x7efedc1436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:28.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.427+0000 7efeda000640 1 -- 172.21.15.88:0/631878532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc15fa40 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efeda000640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efec800d8e0 tx=0x7efec800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8019070 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec80074e0 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80049e0 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc1534e0 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.428+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc162220 con 0x7efedc15f4d0 2024-04-26T20:00:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.429+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc0693c0 con 0x7efedc15f4d0 2024-04-26T20:00:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.430+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efec8004b80 con 0x7efedc15f4d0 2024-04-26T20:00:28.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.431+0000 7efed2a00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:28.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.431+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7efec808b820 con 0x7efedc15f4d0 2024-04-26T20:00:28.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.431+0000 7efedaa00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:28.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.432+0000 7efedaa00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 0x7efebc06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efecc009950 tx=0x7efecc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:28.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.435+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efec8059a90 con 0x7efedc15f4d0 2024-04-26T20:00:28.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:28.714+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) -- 0x7efedc0afd40 con 0x7efedc15f4d0 2024-04-26T20:00:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.148+0000 7efed2a00640 1 -- 172.21.15.88:0/631878532 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v61) ==== 138+0+0 (secure 0 0 0) 0x7efec8059390 con 0x7efedc15f4d0 2024-04-26T20:00:29.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 msgr2=0x7efebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 0x7efebc06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efecc009950 tx=0x7efecc008040 comp rx=0 tx=0).stop 2024-04-26T20:00:29.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 msgr2=0x7efedc161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efec800d8e0 tx=0x7efec800dda0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 shutdown_connections 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efebc06c8f0 0x7efebc06edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efedc142bf0 0x7efedc143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efedc15f4d0 0x7efedc161ce0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 --2- 172.21.15.88:0/631878532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efec0004d40 0x7efedc1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.152+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 >> 172.21.15.88:0/631878532 conn(0x7efedc06c740 msgr2=0x7efedc15e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.153+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 shutdown_connections 2024-04-26T20:00:29.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.153+0000 7efedbe00640 1 -- 172.21.15.88:0/631878532 wait complete. 2024-04-26T20:00:29.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:00:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.371+0000 7fe44e400640 1 Processor -- start 2024-04-26T20:00:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.372+0000 7fe44e400640 1 -- start start 2024-04-26T20:00:29.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe44e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 0x7fe448162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe44e400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4480d3830 con 0x7fe4480b6170 2024-04-26T20:00:29.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe44e400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4480d3b10 con 0x7fe4480d19c0 2024-04-26T20:00:29.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe44e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe4480d40d0 con 0x7fe4480d1da0 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe446a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe4480b6170 0x7fe4480c8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56904/0 (socket says 172.21.15.88:56904) 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.373+0000 7fe446a00640 1 -- 172.21.15.88:0/2772774728 learned_addr learned my addr 172.21.15.88:0/2772774728 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594878510 0 0) 0x7fe4480d3830 con 0x7fe4480b6170 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe428003520 con 0x7fe4480b6170 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe447e00640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 0x7fe448162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1907094212 0 0) 0x7fe428003520 con 0x7fe4480b6170 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 msgr2=0x7fe448162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 0x7fe448162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 >> v1:172.21.15.132:6789/0 conn(0x7fe4480d19c0 legacy=0x7fe4480d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4480d3df0 con 0x7fe4480b6170 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fe4300019f0 con 0x7fe4480b6170 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 >> v1:172.21.15.88:6789/0 conn(0x7fe4480b6170 legacy=0x7fe4480c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 0x7fe428004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe42800bb70 con 0x7fe4280039e0 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe42800bd20 con 0x7fe428004d40 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.374+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe42800bf20 con 0x7fe4480d1da0 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4280039e0 0x7fe428003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447e00640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 -- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 msgr2=0x7fe428004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4480d1da0 0x7fe428004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 -- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe428004d40 msgr2=0x7fe4280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe428004d40 0x7fe4280051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.375+0000 7fe447400640 1 -- 172.21.15.88:0/2772774728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe428003520 con 0x7fe4280039e0 2024-04-26T20:00:29.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.376+0000 7fe447400640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4280039e0 0x7fe428003e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe438001750 tx=0x7fe428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b342731ed233e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:29.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.376+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438001370 con 0x7fe4280039e0 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.376+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4380052d0 con 0x7fe4280039e0 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.376+0000 7fe446000640 1 -- 172.21.15.88:0/2772774728 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43802abe0 con 0x7fe4280039e0 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 -- 172.21.15.88:0/2772774728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4280039e0 msgr2=0x7fe428003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 --2- 172.21.15.88:0/2772774728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4280039e0 0x7fe428003e00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe438001750 tx=0x7fe428005830 comp rx=0 tx=0).stop 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe447e00640 1 -- 172.21.15.88:0/2772774728 reap_dead start 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 -- 172.21.15.88:0/2772774728 shutdown_connections 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 -- 172.21.15.88:0/2772774728 >> 172.21.15.88:0/2772774728 conn(0x7fe44809e460 msgr2=0x7fe4480d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 -- 172.21.15.88:0/2772774728 shutdown_connections 2024-04-26T20:00:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.377+0000 7fe44e400640 1 -- 172.21.15.88:0/2772774728 wait complete. 2024-04-26T20:00:29.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.379+0000 7fe44e400640 1 Processor -- start 2024-04-26T20:00:29.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.379+0000 7fe44e400640 1 -- start start 2024-04-26T20:00:29.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4280039e0 0x7fe4480cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe44815c3d0 0x7fe44815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4481627c0 con 0x7fe448159390 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe448162640 con 0x7fe44815c3d0 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe44e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe448162940 con 0x7fe4280039e0 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe446a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe446a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40300/0 (socket says 172.21.15.88:40300) 2024-04-26T20:00:29.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe446a00640 1 -- 172.21.15.88:0/3655584821 learned_addr learned my addr 172.21.15.88:0/3655584821 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:29.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe447400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4280039e0 0x7fe4480cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe446a00640 1 -- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4280039e0 msgr2=0x7fe4480cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe447e00640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe44815c3d0 0x7fe44815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.380+0000 7fe446a00640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4280039e0 0x7fe4480cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe446a00640 1 -- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe44815c3d0 msgr2=0x7fe44815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe446a00640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe44815c3d0 0x7fe44815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe446a00640 1 -- 172.21.15.88:0/3655584821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe448159810 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe446a00640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe43000cb40 tx=0x7fe43000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430018070 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43000a7d0 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000abc0 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4481582f0 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.381+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481588b0 con 0x7fe448159390 2024-04-26T20:00:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.382+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe448069650 con 0x7fe448159390 2024-04-26T20:00:29.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.383+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe43001b070 con 0x7fe448159390 2024-04-26T20:00:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.384+0000 7fe43f400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 0x7fe42406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.384+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fe43008b410 con 0x7fe448159390 2024-04-26T20:00:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.384+0000 7fe447400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 0x7fe42406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.385+0000 7fe447400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 0x7fe42406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe438032000 tx=0x7fe43802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:29.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.388+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe430059840 con 0x7fe448159390 2024-04-26T20:00:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.589+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe4480d5780 con 0x7fe448159390 2024-04-26T20:00:29.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.594+0000 7fe43f400640 1 -- 172.21.15.88:0/3655584821 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe430059140 con 0x7fe448159390 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 msgr2=0x7fe42406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 0x7fe42406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe438032000 tx=0x7fe43802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 msgr2=0x7fe4480cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe43000cb40 tx=0x7fe43000a060 comp rx=0 tx=0).stop 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 shutdown_connections 2024-04-26T20:00:29.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe44815c3d0 0x7fe44815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe448159390 0x7fe4480cd5f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe42406c920 0x7fe42406ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 --2- 172.21.15.88:0/3655584821 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe4280039e0 0x7fe4480cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.597+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 >> 172.21.15.88:0/3655584821 conn(0x7fe44809e460 msgr2=0x7fe4480d4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.598+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 shutdown_connections 2024-04-26T20:00:29.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:29.598+0000 7fe44e400640 1 -- 172.21.15.88:0/3655584821 wait complete. 2024-04-26T20:00:29.618 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:00:29.618 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:00:29.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:00:29.831 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:00:29.831 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:29.832 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:29.889 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:29.889 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:29.940 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.124+0000 7fa776400640 1 Processor -- start 2024-04-26T20:00:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.125+0000 7fa776400640 1 -- start start 2024-04-26T20:00:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.126+0000 7fa776400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa77015dc80 0x7fa770167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.126+0000 7fa776400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.126+0000 7fa776400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa770164740 con 0x7fa77006c770 2024-04-26T20:00:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.126+0000 7fa776400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa7700b5ff0 con 0x7fa77015dc80 2024-04-26T20:00:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.126+0000 7fa776400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa77006d110 con 0x7fa770164240 2024-04-26T20:00:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.127+0000 7fa76f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa77015dc80 0x7fa770167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.127+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.127+0000 7fa76f400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa77015dc80 0x7fa770167c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:40508/0 (socket says 172.21.15.88:40508) 2024-04-26T20:00:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.127+0000 7fa76f400640 1 -- 172.21.15.88:0/2429011056 learned_addr learned my addr 172.21.15.88:0/2429011056 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.128+0000 7fa76f400640 1 -- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 msgr2=0x7fa77006cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.128+0000 7fa76f400640 1 --2- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.128+0000 7fa76f400640 1 -- 172.21.15.88:0/2429011056 >> v1:172.21.15.132:6789/0 conn(0x7fa77006c770 legacy=0x7fa770163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.128+0000 7fa76f400640 1 -- 172.21.15.88:0/2429011056 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7701671a0 con 0x7fa77015dc80 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.128+0000 7fa76ea00640 1 -- 172.21.15.88:0/2429011056 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809087014 0 0) 0x7fa770164740 con 0x7fa77006c770 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.130+0000 7fa76f400640 1 --2- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa770167c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa760009e80 tx=0x7fa76002a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3358f562dd494f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.130+0000 7fa76ea00640 1 -- 172.21.15.88:0/2429011056 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760036070 con 0x7fa77015dc80 2024-04-26T20:00:30.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.130+0000 7fa76ea00640 1 -- 172.21.15.88:0/2429011056 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760031600 con 0x7fa77015dc80 2024-04-26T20:00:30.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.130+0000 7fa76ea00640 1 -- 172.21.15.88:0/2429011056 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7600319d0 con 0x7fa77015dc80 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 -- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 msgr2=0x7fa770167c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 --2- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa770167c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa760009e80 tx=0x7fa76002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 -- 172.21.15.88:0/2429011056 shutdown_connections 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 --2- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 --2- 172.21.15.88:0/2429011056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa770167c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 -- 172.21.15.88:0/2429011056 >> 172.21.15.88:0/2429011056 conn(0x7fa7700b2d50 msgr2=0x7fa7700b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 -- 172.21.15.88:0/2429011056 shutdown_connections 2024-04-26T20:00:30.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.131+0000 7fa776400640 1 -- 172.21.15.88:0/2429011056 wait complete. 2024-04-26T20:00:30.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.133+0000 7fa776400640 1 Processor -- start 2024-04-26T20:00:30.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.133+0000 7fa776400640 1 -- start start 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa77006ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa770164a20 con 0x7fa77006c770 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa770167760 con 0x7fa77015dc80 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa776400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa7700add80 con 0x7fa770164240 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40312/0 (socket says 172.21.15.88:40312) 2024-04-26T20:00:30.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 -- 172.21.15.88:0/156695996 learned_addr learned my addr 172.21.15.88:0/156695996 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa766a00640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa77006ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76fe00640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 -- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 msgr2=0x7fa77006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 -- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 msgr2=0x7fa77006ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.134+0000 7fa76f400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa77006ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa76f400640 1 -- 172.21.15.88:0/156695996 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77009db10 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa76f400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa760034040 tx=0x7fa760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760036050 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760037040 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760009510 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77009ddf0 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.135+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77009e3b0 con 0x7fa77006c770 2024-04-26T20:00:30.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.136+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770069540 con 0x7fa77006c770 2024-04-26T20:00:30.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.137+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa760044020 con 0x7fa77006c770 2024-04-26T20:00:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.138+0000 7fa767e00640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 0x7fa74c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.138+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fa7600ac0c0 con 0x7fa77006c770 2024-04-26T20:00:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.139+0000 7fa766a00640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 0x7fa74c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.139+0000 7fa766a00640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 0x7fa74c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7700ccfa0 tx=0x7fa758009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.143+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa76004b020 con 0x7fa77006c770 2024-04-26T20:00:30.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.344+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa77009eeb0 con 0x7fa77006c770 2024-04-26T20:00:30.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.346+0000 7fa767e00640 1 -- 172.21.15.88:0/156695996 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7fa7600a3e20 con 0x7fa77006c770 2024-04-26T20:00:30.349 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:30.349 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:30.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 msgr2=0x7fa74c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 0x7fa74c06ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7700ccfa0 tx=0x7fa758009040 comp rx=0 tx=0).stop 2024-04-26T20:00:30.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 msgr2=0x7fa77009d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa760034040 tx=0x7fa760004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:30.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 shutdown_connections 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa74c06c920 0x7fa74c06ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa770164240 0x7fa77006f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77015dc80 0x7fa77006ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 --2- 172.21.15.88:0/156695996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77006c770 0x7fa77009d3a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 >> 172.21.15.88:0/156695996 conn(0x7fa7700b2d50 msgr2=0x7fa770164ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.349+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 shutdown_connections 2024-04-26T20:00:30.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.350+0000 7fa776400640 1 -- 172.21.15.88:0/156695996 wait complete. 2024-04-26T20:00:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:30.370 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:30.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.568+0000 7f3c1b200640 1 Processor -- start 2024-04-26T20:00:30.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.568+0000 7f3c1b200640 1 -- start start 2024-04-26T20:00:30.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 0x7f3c1c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c1b200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c0b4d20 con 0x7f3c1c161140 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c1b200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c0b5000 con 0x7f3c1c160d60 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c06e5f0 con 0x7f3c1c146280 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c19400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3c1c161140 0x7f3c1c06dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56912/0 (socket says 172.21.15.88:56912) 2024-04-26T20:00:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.570+0000 7f3c19400640 1 -- 172.21.15.88:0/3960989964 learned_addr learned my addr 172.21.15.88:0/3960989964 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:30.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714157766 0 0) 0x7f3c1c0b4d20 con 0x7f3c1c161140 2024-04-26T20:00:30.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c00003520 con 0x7f3c1c161140 2024-04-26T20:00:30.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c1a800640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 0x7f3c1c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2258561165 0 0) 0x7f3c00003520 con 0x7f3c1c161140 2024-04-26T20:00:30.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 msgr2=0x7f3c1c1466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 0x7f3c1c1466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 >> v1:172.21.15.132:6789/0 conn(0x7f3c1c160d60 legacy=0x7f3c1c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0b52e0 con 0x7f3c1c161140 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f3c08001670 con 0x7f3c1c161140 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 >> v1:172.21.15.88:6789/0 conn(0x7f3c1c161140 legacy=0x7f3c1c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 0x7f3c00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c00004d90 0x7f3c00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c06e5f0 con 0x7f3c00003a50 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0000bb80 con 0x7f3c00004d90 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0000bd80 con 0x7f3c1c146280 2024-04-26T20:00:30.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.571+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c1a800640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c00004d90 0x7f3c00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c19e00640 1 -- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 msgr2=0x7f3c00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c146280 0x7f3c00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c19e00640 1 -- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c00004d90 msgr2=0x7f3c00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c00004d90 0x7f3c00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c19e00640 1 -- 172.21.15.88:0/3960989964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c00003520 con 0x7f3c00003a50 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.572+0000 7f3c1a800640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c00004d90 0x7f3c00005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:30.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.573+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c00003e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c001750 tx=0x7f3c00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5682563433cb2803 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.573+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c001370 con 0x7f3c00003a50 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.573+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0052d0 con 0x7f3c00003a50 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.573+0000 7f3c18a00640 1 -- 172.21.15.88:0/3960989964 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c02abe0 con 0x7f3c00003a50 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 -- 172.21.15.88:0/3960989964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 msgr2=0x7f3c00003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3960989964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c00003e70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c001750 tx=0x7f3c00005830 comp rx=0 tx=0).stop 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1a800640 1 -- 172.21.15.88:0/3960989964 reap_dead start 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 -- 172.21.15.88:0/3960989964 shutdown_connections 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 -- 172.21.15.88:0/3960989964 >> 172.21.15.88:0/3960989964 conn(0x7f3c1c0b2900 msgr2=0x7f3c1c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:30.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 -- 172.21.15.88:0/3960989964 shutdown_connections 2024-04-26T20:00:30.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.574+0000 7f3c1b200640 1 -- 172.21.15.88:0/3960989964 wait complete. 2024-04-26T20:00:30.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.576+0000 7f3c1b200640 1 Processor -- start 2024-04-26T20:00:30.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.576+0000 7f3c1b200640 1 -- start start 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c1c159950 0x7f3c1c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c072500 0x7f3c1c072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c0cf6f0 con 0x7f3c00003a50 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c146c30 con 0x7f3c1c159950 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c1b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c161770 con 0x7f3c1c072500 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40324/0 (socket says 172.21.15.88:40324) 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 -- 172.21.15.88:0/3601690863 learned_addr learned my addr 172.21.15.88:0/3601690863 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 -- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c072500 msgr2=0x7f3c1c072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c072500 0x7f3c1c072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 -- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c1c159950 msgr2=0x7f3c1c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c1c159950 0x7f3c1c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.577+0000 7f3c19e00640 1 -- 172.21.15.88:0/3601690863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c0b3ad0 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c19e00640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c001870 tx=0x7f3c0c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c037040 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c005480 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c007a30 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c1f85a0 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.578+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1f8b60 con 0x7f3c00003a50 2024-04-26T20:00:30.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.579+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c0693c0 con 0x7f3c00003a50 2024-04-26T20:00:30.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.580+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c0c044020 con 0x7f3c00003a50 2024-04-26T20:00:30.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.582+0000 7f3c12000640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 0x7f3bfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:30.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.582+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f3c0c0ab820 con 0x7f3c00003a50 2024-04-26T20:00:30.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.582+0000 7f3c19400640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 0x7f3bfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:30.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.583+0000 7f3c19400640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 0x7f3bfc06ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06eba0 tx=0x7f3c08001ac0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:30.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.586+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c0c079bb0 con 0x7f3c00003a50 2024-04-26T20:00:30.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.780+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c1c145ba0 con 0x7f3c00003a50 2024-04-26T20:00:30.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.781+0000 7f3c12000640 1 -- 172.21.15.88:0/3601690863 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f3c0c0794b0 con 0x7f3c00003a50 2024-04-26T20:00:30.783 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:30.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.783+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 msgr2=0x7f3bfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.783+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 0x7f3bfc06ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c06eba0 tx=0x7f3c08001ac0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.783+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 msgr2=0x7f3c1c0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:30.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.783+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c001870 tx=0x7f3c0c001120 comp rx=0 tx=0).stop 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 shutdown_connections 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3bfc06c920 0x7f3bfc06ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c1c072500 0x7f3c1c072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c1c159950 0x7f3c1c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 --2- 172.21.15.88:0/3601690863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c00003a50 0x7f3c1c0b3590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 >> 172.21.15.88:0/3601690863 conn(0x7f3c1c0b2900 msgr2=0x7f3c1c0b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 shutdown_connections 2024-04-26T20:00:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.784+0000 7f3c1b200640 1 -- 172.21.15.88:0/3601690863 wait complete. 2024-04-26T20:00:30.805 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:31.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.998+0000 7f3059800640 1 Processor -- start 2024-04-26T20:00:31.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.998+0000 7f3059800640 1 -- start start 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:30.999+0000 7f3059800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f305406ce80 0x7f30540b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3059800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f30540b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3059800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30540af800 con 0x7f305415d100 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3059800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f30540ae340 con 0x7f305406ce80 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3059800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f30540afaa0 con 0x7f30540b5ca0 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3052000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f305406ce80 0x7f30540b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3052000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f305406ce80 0x7f30540b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40338/0 (socket says 172.21.15.88:40338) 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3052000640 1 -- 172.21.15.88:0/2923538864 learned_addr learned my addr 172.21.15.88:0/2923538864 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.000+0000 7f3053400640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f30540b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.001+0000 7f3052000640 1 -- 172.21.15.88:0/2923538864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 msgr2=0x7f30540b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.001+0000 7f3052000640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f30540b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.001+0000 7f3052000640 1 -- 172.21.15.88:0/2923538864 >> v1:172.21.15.132:6789/0 conn(0x7f305415d100 legacy=0x7f305406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.001+0000 7f3052000640 1 -- 172.21.15.88:0/2923538864 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30540afd80 con 0x7f305406ce80 2024-04-26T20:00:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.001+0000 7f3053400640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f30540b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:31.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.002+0000 7f3052000640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305406ce80 0x7f30540b3c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3040009870 tx=0x7f304002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f44b4e0f28616c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.002+0000 7f3051600640 1 -- 172.21.15.88:0/2923538864 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040037070 con 0x7f305406ce80 2024-04-26T20:00:31.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.002+0000 7f3051600640 1 -- 172.21.15.88:0/2923538864 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040032440 con 0x7f305406ce80 2024-04-26T20:00:31.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.002+0000 7f3051600640 1 -- 172.21.15.88:0/2923538864 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040032810 con 0x7f305406ce80 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 -- 172.21.15.88:0/2923538864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305406ce80 msgr2=0x7f30540b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305406ce80 0x7f30540b3c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3040009870 tx=0x7f304002a270 comp rx=0 tx=0).stop 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 -- 172.21.15.88:0/2923538864 shutdown_connections 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f30540b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 --2- 172.21.15.88:0/2923538864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305406ce80 0x7f30540b3c30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 -- 172.21.15.88:0/2923538864 >> 172.21.15.88:0/2923538864 conn(0x7f30540b2d50 msgr2=0x7f30540b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 -- 172.21.15.88:0/2923538864 shutdown_connections 2024-04-26T20:00:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.003+0000 7f3059800640 1 -- 172.21.15.88:0/2923538864 wait complete. 2024-04-26T20:00:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.005+0000 7f3059800640 1 Processor -- start 2024-04-26T20:00:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.005+0000 7f3059800640 1 -- start start 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f305406ce80 0x7f3054151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f3054156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f305409ee50 con 0x7f305415d100 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f30540b0340 con 0x7f305406ce80 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3059800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3054160b70 con 0x7f30540b5ca0 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3053400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3053400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40342/0 (socket says 172.21.15.88:40342) 2024-04-26T20:00:31.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.006+0000 7f3053400640 1 -- 172.21.15.88:0/4184031717 learned_addr learned my addr 172.21.15.88:0/4184031717 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3052a00640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f305406ce80 0x7f3054151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3052000640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f3054156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 -- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 msgr2=0x7f3054156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f3054156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 -- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f305406ce80 msgr2=0x7f3054151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f305406ce80 0x7f3054151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 -- 172.21.15.88:0/4184031717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305406f960 con 0x7f305415d100 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3053400640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f304c009fa0 tx=0x7f304c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c01a070 con 0x7f305415d100 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304c003bb0 con 0x7f305415d100 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c004030 con 0x7f305415d100 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305406fc40 con 0x7f305415d100 2024-04-26T20:00:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.007+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3054151b50 con 0x7f305415d100 2024-04-26T20:00:31.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.008+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30540b1d90 con 0x7f305415d100 2024-04-26T20:00:31.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.009+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f304c01d070 con 0x7f305415d100 2024-04-26T20:00:31.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.011+0000 7f304aa00640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 0x7f303406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.011+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f304c08bf00 con 0x7f305415d100 2024-04-26T20:00:31.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.011+0000 7f3052a00640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 0x7f303406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.012+0000 7f3052a00640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 0x7f303406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3044007fd0 tx=0x7f3044009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.016+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30540b1d90 con 0x7f305415d100 2024-04-26T20:00:31.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.200+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3054069650 con 0x7f305415d100 2024-04-26T20:00:31.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.201+0000 7f304aa00640 1 -- 172.21.15.88:0/4184031717 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f304c010070 con 0x7f305415d100 2024-04-26T20:00:31.203 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.203+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 msgr2=0x7f303406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.203+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 0x7f303406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3044007fd0 tx=0x7f3044009040 comp rx=0 tx=0).stop 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 msgr2=0x7f30541571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f304c009fa0 tx=0x7f304c00d560 comp rx=0 tx=0).stop 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 shutdown_connections 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f305415d100 0x7f30541571b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f303406c8f0 0x7f303406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f30540b5ca0 0x7f3054156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 --2- 172.21.15.88:0/4184031717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f305406ce80 0x7f3054151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 >> 172.21.15.88:0/4184031717 conn(0x7f30540b2d50 msgr2=0x7f30540b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:31.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 shutdown_connections 2024-04-26T20:00:31.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.204+0000 7f3059800640 1 -- 172.21.15.88:0/4184031717 wait complete. 2024-04-26T20:00:31.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:31.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.419+0000 7f9856400640 1 Processor -- start 2024-04-26T20:00:31.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.419+0000 7f9856400640 1 -- start start 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f9856400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850147f20 0x7f98500b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f9856400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9850073da0 0x7f985006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f9856400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98500afed0 con 0x7f985006ccb0 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f9856400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f985006d0e0 con 0x7f9850073da0 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f9856400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f98500ade20 con 0x7f9850147f20 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f984ea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9850073da0 0x7f985006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f984ea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9850073da0 0x7f985006c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40356/0 (socket says 172.21.15.88:40356) 2024-04-26T20:00:31.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.421+0000 7f984ea00640 1 -- 172.21.15.88:0/323732482 learned_addr learned my addr 172.21.15.88:0/323732482 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.422+0000 7f984ea00640 1 -- 172.21.15.88:0/323732482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850147f20 msgr2=0x7f98500b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:31.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.422+0000 7f984ea00640 1 --2- 172.21.15.88:0/323732482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850147f20 0x7f98500b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.422+0000 7f984ea00640 1 -- 172.21.15.88:0/323732482 >> v1:172.21.15.132:6789/0 conn(0x7f985006ccb0 legacy=0x7f9850072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:31.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.422+0000 7f984ea00640 1 -- 172.21.15.88:0/323732482 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98500ae100 con 0x7f9850073da0 2024-04-26T20:00:31.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.423+0000 7f984ea00640 1 --2- 172.21.15.88:0/323732482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850073da0 0x7f985006c770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f983c009870 tx=0x7f983c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3140476622e057 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.423+0000 7f984e000640 1 -- 172.21.15.88:0/323732482 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c037070 con 0x7f9850073da0 2024-04-26T20:00:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.423+0000 7f984e000640 1 -- 172.21.15.88:0/323732482 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983c032440 con 0x7f9850073da0 2024-04-26T20:00:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.423+0000 7f984e000640 1 -- 172.21.15.88:0/323732482 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c032810 con 0x7f9850073da0 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 -- 172.21.15.88:0/323732482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850073da0 msgr2=0x7f985006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 --2- 172.21.15.88:0/323732482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850073da0 0x7f985006c770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f983c009870 tx=0x7f983c02a210 comp rx=0 tx=0).stop 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 -- 172.21.15.88:0/323732482 shutdown_connections 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 --2- 172.21.15.88:0/323732482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850073da0 0x7f985006c770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 --2- 172.21.15.88:0/323732482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9850147f20 0x7f98500b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 -- 172.21.15.88:0/323732482 >> 172.21.15.88:0/323732482 conn(0x7f98500b2d50 msgr2=0x7f98500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 -- 172.21.15.88:0/323732482 shutdown_connections 2024-04-26T20:00:31.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.424+0000 7f9856400640 1 -- 172.21.15.88:0/323732482 wait complete. 2024-04-26T20:00:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.426+0000 7f9856400640 1 Processor -- start 2024-04-26T20:00:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.426+0000 7f9856400640 1 -- start start 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985006ccb0 0x7f98500d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850073da0 0x7f9850161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98500ae9a0 con 0x7f9850147f20 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f98500ae6c0 con 0x7f9850073da0 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f9856400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9850074200 con 0x7f985006ccb0 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f984fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f984fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40360/0 (socket says 172.21.15.88:40360) 2024-04-26T20:00:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.427+0000 7f984fe00640 1 -- 172.21.15.88:0/1836951394 learned_addr learned my addr 172.21.15.88:0/1836951394 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 -- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985006ccb0 msgr2=0x7f98500d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985006ccb0 0x7f98500d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 -- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850073da0 msgr2=0x7f9850161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850073da0 0x7f9850161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 -- 172.21.15.88:0/1836951394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9850162490 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f984fe00640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f984800b9c0 tx=0x7f984800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848019070 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984800cba0 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848014440 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9850161100 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.428+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98501616c0 con 0x7f9850147f20 2024-04-26T20:00:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.430+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9850069650 con 0x7f9850147f20 2024-04-26T20:00:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.431+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9848003970 con 0x7f9850147f20 2024-04-26T20:00:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.431+0000 7f9847400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 0x7f983006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.432+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f984808b7e0 con 0x7f9850147f20 2024-04-26T20:00:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.432+0000 7f984f400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 0x7f983006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.432+0000 7f984f400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 0x7f983006ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9840009710 tx=0x7f9840006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.436+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9850162490 con 0x7f9850147f20 2024-04-26T20:00:31.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.636+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f98500b3130 con 0x7f9850147f20 2024-04-26T20:00:31.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.637+0000 7f9847400640 1 -- 172.21.15.88:0/1836951394 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f9848059b70 con 0x7f9850147f20 2024-04-26T20:00:31.639 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.639+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 msgr2=0x7f983006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.639+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 0x7f983006ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9840009710 tx=0x7f9840006c90 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.639+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 msgr2=0x7f98500cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.639+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f984800b9c0 tx=0x7f984800be80 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 shutdown_connections 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9850147f20 0x7f98500cc110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9850073da0 0x7f9850161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f983006c920 0x7f983006ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 --2- 172.21.15.88:0/1836951394 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f985006ccb0 0x7f98500d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 >> 172.21.15.88:0/1836951394 conn(0x7f98500b2d50 msgr2=0x7f98500d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 shutdown_connections 2024-04-26T20:00:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.640+0000 7f9856400640 1 -- 172.21.15.88:0/1836951394 wait complete. 2024-04-26T20:00:31.659 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:31.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.849+0000 7fb947e00640 1 Processor -- start 2024-04-26T20:00:31.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.849+0000 7fb947e00640 1 -- start start 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb947e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 0x7fb94006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb947e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb94015cb70 0x7fb940073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb947e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94006d090 con 0x7fb94015cfd0 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb947e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb9400af460 con 0x7fb94015cb70 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb947e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb9400ae850 con 0x7fb94006c770 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb944a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb94015cb70 0x7fb940073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb944a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb94015cb70 0x7fb940073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40362/0 (socket says 172.21.15.88:40362) 2024-04-26T20:00:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.851+0000 7fb944a00640 1 -- 172.21.15.88:0/3607897914 learned_addr learned my addr 172.21.15.88:0/3607897914 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb944a00640 1 -- 172.21.15.88:0/3607897914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 msgr2=0x7fb94006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb945400640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 0x7fb94006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb944a00640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 0x7fb94006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb944a00640 1 -- 172.21.15.88:0/3607897914 >> v1:172.21.15.132:6789/0 conn(0x7fb94015cfd0 legacy=0x7fb940073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb944a00640 1 -- 172.21.15.88:0/3607897914 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400aeb30 con 0x7fb94015cb70 2024-04-26T20:00:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.852+0000 7fb945400640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 0x7fb94006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:31.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.853+0000 7fb944a00640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94015cb70 0x7fb940073120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009870 tx=0x7fb92c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec501eab4ee3025 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.853+0000 7fb93fe00640 1 -- 172.21.15.88:0/3607897914 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c037070 con 0x7fb94015cb70 2024-04-26T20:00:31.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.853+0000 7fb93fe00640 1 -- 172.21.15.88:0/3607897914 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92c032440 con 0x7fb94015cb70 2024-04-26T20:00:31.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.853+0000 7fb93fe00640 1 -- 172.21.15.88:0/3607897914 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c032810 con 0x7fb94015cb70 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 -- 172.21.15.88:0/3607897914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94015cb70 msgr2=0x7fb940073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94015cb70 0x7fb940073120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009870 tx=0x7fb92c02a210 comp rx=0 tx=0).stop 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 -- 172.21.15.88:0/3607897914 shutdown_connections 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94015cb70 0x7fb940073120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 --2- 172.21.15.88:0/3607897914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94006c770 0x7fb94006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 -- 172.21.15.88:0/3607897914 >> 172.21.15.88:0/3607897914 conn(0x7fb9400b2d50 msgr2=0x7fb9400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:31.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 -- 172.21.15.88:0/3607897914 shutdown_connections 2024-04-26T20:00:31.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.854+0000 7fb947e00640 1 -- 172.21.15.88:0/3607897914 wait complete. 2024-04-26T20:00:31.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.856+0000 7fb947e00640 1 Processor -- start 2024-04-26T20:00:31.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.857+0000 7fb947e00640 1 -- start start 2024-04-26T20:00:31.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94015cb70 0x7fb9400cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb94015cfd0 0x7fb94009e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb94015c600 con 0x7fb94006c770 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb9400af0f0 con 0x7fb94015cfd0 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb947e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb94015c780 con 0x7fb94015cb70 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40364/0 (socket says 172.21.15.88:40364) 2024-04-26T20:00:31.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 -- 172.21.15.88:0/2670498218 learned_addr learned my addr 172.21.15.88:0/2670498218 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:31.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 -- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94015cb70 msgr2=0x7fb9400cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94015cb70 0x7fb9400cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945400640 1 -- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb94015cfd0 msgr2=0x7fb94009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:31.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.858+0000 7fb945e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb94015cfd0 0x7fb94009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb945400640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb94015cfd0 0x7fb94009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb945400640 1 -- 172.21.15.88:0/2670498218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400cf7f0 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb945400640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c820 tx=0x7fb93400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934015070 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb934007a30 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934007e00 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9400cfad0 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.859+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9400d0090 con 0x7fb94006c770 2024-04-26T20:00:31.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.860+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb940069540 con 0x7fb94006c770 2024-04-26T20:00:31.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.861+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb934018070 con 0x7fb94006c770 2024-04-26T20:00:31.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.862+0000 7fb93d600640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 0x7fb92006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:31.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.862+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fb93408bbb0 con 0x7fb94006c770 2024-04-26T20:00:31.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.862+0000 7fb944a00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 0x7fb92006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:31.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.863+0000 7fb944a00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 0x7fb92006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb92c034000 tx=0x7fb92c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:31.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:31.867+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb934059f50 con 0x7fb94006c770 2024-04-26T20:00:32.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.065+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9400cd430 con 0x7fb94006c770 2024-04-26T20:00:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.066+0000 7fb93d600640 1 -- 172.21.15.88:0/2670498218 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb934059850 con 0x7fb94006c770 2024-04-26T20:00:32.067 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 msgr2=0x7fb92006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 0x7fb92006edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb92c034000 tx=0x7fb92c02c000 comp rx=0 tx=0).stop 2024-04-26T20:00:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 msgr2=0x7fb94014b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c820 tx=0x7fb93400cc20 comp rx=0 tx=0).stop 2024-04-26T20:00:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 shutdown_connections 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb94015cfd0 0x7fb94009e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb92006c8f0 0x7fb92006edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb94015cb70 0x7fb9400cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.068+0000 7fb947e00640 1 --2- 172.21.15.88:0/2670498218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94006c770 0x7fb94014b2b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.069+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 >> 172.21.15.88:0/2670498218 conn(0x7fb9400b2d50 msgr2=0x7fb94014bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.069+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 shutdown_connections 2024-04-26T20:00:32.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.069+0000 7fb947e00640 1 -- 172.21.15.88:0/2670498218 wait complete. 2024-04-26T20:00:32.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:32.088 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:32.134 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:32.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.375+0000 7f9ce8a00640 1 Processor -- start 2024-04-26T20:00:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.375+0000 7f9ce8a00640 1 -- start start 2024-04-26T20:00:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce8a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9ce40d1580 0x7f9ce415f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce8a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce4073110 con 0x7f9ce4145570 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce40733f0 con 0x7f9ce4073ed0 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce8a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce40cef40 con 0x7f9ce40d1580 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9ce40d1580 0x7f9ce415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9ce40d1580 0x7f9ce415f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40380/0 (socket says 172.21.15.88:40380) 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce2800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9ce4145570 0x7f9ce40d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56926/0 (socket says 172.21.15.88:56926) 2024-04-26T20:00:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 -- 172.21.15.88:0/4134206509 learned_addr learned my addr 172.21.15.88:0/4134206509 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 -- 172.21.15.88:0/4134206509 >> v1:172.21.15.132:6789/0 conn(0x7f9ce4073ed0 legacy=0x7f9ce415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 -- 172.21.15.88:0/4134206509 >> v1:172.21.15.88:6789/0 conn(0x7f9ce4145570 legacy=0x7f9ce40d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9cd9e00640 1 -- 172.21.15.88:0/4134206509 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce40736d0 con 0x7f9ce40d1580 2024-04-26T20:00:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.377+0000 7f9ce1e00640 1 -- 172.21.15.88:0/4134206509 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072836406 0 0) 0x7f9ce4073110 con 0x7f9ce4145570 2024-04-26T20:00:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9cd9e00640 1 --2- 172.21.15.88:0/4134206509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415f940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00bb10 tx=0x7f9ccc02c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb89fdd972cd0ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9ce1e00640 1 -- 172.21.15.88:0/4134206509 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc032790 con 0x7f9ce40d1580 2024-04-26T20:00:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9ce1e00640 1 -- 172.21.15.88:0/4134206509 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ccc038070 con 0x7f9ce40d1580 2024-04-26T20:00:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9ce1e00640 1 -- 172.21.15.88:0/4134206509 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc032b50 con 0x7f9ce40d1580 2024-04-26T20:00:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9ce8a00640 1 -- 172.21.15.88:0/4134206509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 msgr2=0x7f9ce415f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.379+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/4134206509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415f940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00bb10 tx=0x7f9ccc02c8d0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.380+0000 7f9ce8a00640 1 -- 172.21.15.88:0/4134206509 shutdown_connections 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.380+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/4134206509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415f940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.380+0000 7f9ce8a00640 1 -- 172.21.15.88:0/4134206509 >> 172.21.15.88:0/4134206509 conn(0x7f9ce406c740 msgr2=0x7f9ce406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.380+0000 7f9ce8a00640 1 -- 172.21.15.88:0/4134206509 shutdown_connections 2024-04-26T20:00:32.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.380+0000 7f9ce8a00640 1 -- 172.21.15.88:0/4134206509 wait complete. 2024-04-26T20:00:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.381+0000 7f9ce8a00640 1 Processor -- start 2024-04-26T20:00:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.381+0000 7f9ce8a00640 1 -- start start 2024-04-26T20:00:32.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 0x7f9ce415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 0x7f9ce415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce4145950 con 0x7f9ce40d1580 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce41606b0 con 0x7f9ce4073ed0 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.382+0000 7f9ce8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce406dbc0 con 0x7f9ce4145570 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40392/0 (socket says 172.21.15.88:40392) 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 -- 172.21.15.88:0/2071272215 learned_addr learned my addr 172.21.15.88:0/2071272215 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9ce2800640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 0x7f9ce415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9ce3200640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 0x7f9ce415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 -- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 msgr2=0x7f9ce415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 0x7f9ce415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 -- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 msgr2=0x7f9ce415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 0x7f9ce415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9cd9e00640 1 -- 172.21.15.88:0/2071272215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce415e750 con 0x7f9ce40d1580 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9ce2800640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 0x7f9ce415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.383+0000 7f9ce3200640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 0x7f9ce415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9cd9e00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00bb10 tx=0x7f9ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc032e60 con 0x7f9ce40d1580 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ccc033850 con 0x7f9ce40d1580 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc033c40 con 0x7f9ce40d1580 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce415e9f0 con 0x7f9ce40d1580 2024-04-26T20:00:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.384+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce415cdf0 con 0x7f9ce40d1580 2024-04-26T20:00:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.385+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce415e4e0 con 0x7f9ce40d1580 2024-04-26T20:00:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.386+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ccc04e3f0 con 0x7f9ce40d1580 2024-04-26T20:00:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.387+0000 7f9cdb400640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 0x7f9cc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.387+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f9ccc0aea90 con 0x7f9ce40d1580 2024-04-26T20:00:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.387+0000 7f9ce2800640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 0x7f9cc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.388+0000 7f9ce2800640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 0x7f9cc006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f60 tx=0x7f9cd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.391+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ccc041070 con 0x7f9ce40d1580 2024-04-26T20:00:32.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.590+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ce40693c0 con 0x7f9ce40d1580 2024-04-26T20:00:32.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.592+0000 7f9cdb400640 1 -- 172.21.15.88:0/2071272215 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f9ccc07c740 con 0x7f9ce40d1580 2024-04-26T20:00:32.595 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:32.595 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 msgr2=0x7f9cc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 0x7f9cc006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f60 tx=0x7f9cd4009040 comp rx=0 tx=0).stop 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 msgr2=0x7f9ce415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00bb10 tx=0x7f9ccc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 shutdown_connections 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9cc006c8f0 0x7f9cc006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ce4145570 0x7f9ce415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ce40d1580 0x7f9ce415afb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 --2- 172.21.15.88:0/2071272215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ce4073ed0 0x7f9ce415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 >> 172.21.15.88:0/2071272215 conn(0x7f9ce406c740 msgr2=0x7f9ce406cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 shutdown_connections 2024-04-26T20:00:32.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.595+0000 7f9ce8a00640 1 -- 172.21.15.88:0/2071272215 wait complete. 2024-04-26T20:00:32.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:32.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:32.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.802+0000 7f1c4a600640 1 Processor -- start 2024-04-26T20:00:32.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.802+0000 7f1c4a600640 1 -- start start 2024-04-26T20:00:32.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c440b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c4a600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4415d970 con 0x7f1c4415ee60 2024-04-26T20:00:32.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c4a600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c440b4fe0 con 0x7f1c4406dfe0 2024-04-26T20:00:32.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c4a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1c440b52c0 con 0x7f1c4415c450 2024-04-26T20:00:32.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c49c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1c4415ee60 0x7f1c44165a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56942/0 (socket says 172.21.15.88:56942) 2024-04-26T20:00:32.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c49c00640 1 -- 172.21.15.88:0/1266800285 learned_addr learned my addr 172.21.15.88:0/1266800285 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.804+0000 7f1c49200640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c440b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827455349 0 0) 0x7f1c4415d970 con 0x7f1c4415ee60 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c2c005730 con 0x7f1c4415ee60 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2175644158 0 0) 0x7f1c2c005730 con 0x7f1c4415ee60 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 msgr2=0x7f1c440b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c440b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 >> v1:172.21.15.132:6789/0 conn(0x7f1c4406dfe0 legacy=0x7f1c4415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c440b55a0 con 0x7f1c4415ee60 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f1c3c001a10 con 0x7f1c4415ee60 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 >> v1:172.21.15.88:6789/0 conn(0x7f1c4415ee60 legacy=0x7f1c44165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c2c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c2c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c006f50 0x7f1c2c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440b52c0 con 0x7f1c2c005c60 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c00dd60 con 0x7f1c2c006f50 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c00df60 con 0x7f1c4415c450 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.805+0000 7f1c49200640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c2c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49c00640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c006f50 0x7f1c2c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c43e00640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c2c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49200640 1 -- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 msgr2=0x7f1c2c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:32.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49200640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c2c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49200640 1 -- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c006f50 msgr2=0x7f1c2c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49200640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c006f50 0x7f1c2c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49200640 1 -- 172.21.15.88:0/1266800285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c005730 con 0x7f1c2c005c60 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c49c00640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c006f50 0x7f1c2c0073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.806+0000 7f1c43e00640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c4415c450 0x7f1c2c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.807+0000 7f1c49200640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c2c006080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009ac0 tx=0x7f1c2c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3d583497e16da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.807+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38036070 con 0x7f1c2c005c60 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.807+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c38031400 con 0x7f1c2c005c60 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.807+0000 7f1c43400640 1 -- 172.21.15.88:0/1266800285 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38009690 con 0x7f1c2c005c60 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 -- 172.21.15.88:0/1266800285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 msgr2=0x7f1c2c006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 --2- 172.21.15.88:0/1266800285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c2c006080 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009ac0 tx=0x7f1c2c007a40 comp rx=0 tx=0).stop 2024-04-26T20:00:32.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c49c00640 1 -- 172.21.15.88:0/1266800285 reap_dead start 2024-04-26T20:00:32.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 -- 172.21.15.88:0/1266800285 shutdown_connections 2024-04-26T20:00:32.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 -- 172.21.15.88:0/1266800285 >> 172.21.15.88:0/1266800285 conn(0x7f1c440b2900 msgr2=0x7f1c440b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:32.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 -- 172.21.15.88:0/1266800285 shutdown_connections 2024-04-26T20:00:32.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.808+0000 7f1c4a600640 1 -- 172.21.15.88:0/1266800285 wait complete. 2024-04-26T20:00:32.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.810+0000 7f1c4a600640 1 Processor -- start 2024-04-26T20:00:32.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.810+0000 7f1c4a600640 1 -- start start 2024-04-26T20:00:32.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c440ce160 0x7f1c440cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4414aa60 0x7f1c44072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440b5a00 con 0x7f1c2c005c60 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440b5880 con 0x7f1c4414aa60 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c4a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4415cf40 con 0x7f1c440ce160 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c49200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c49200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40412/0 (socket says 172.21.15.88:40412) 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c49200640 1 -- 172.21.15.88:0/3755046103 learned_addr learned my addr 172.21.15.88:0/3755046103 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.811+0000 7f1c43e00640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c440ce160 0x7f1c440cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 -- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c440ce160 msgr2=0x7f1c440cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49c00640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4414aa60 0x7f1c44072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c440ce160 0x7f1c440cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 -- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4414aa60 msgr2=0x7f1c44072ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4414aa60 0x7f1c44072ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 -- 172.21.15.88:0/3755046103 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c440ce7d0 con 0x7f1c2c005c60 2024-04-26T20:00:32.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c49200640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c38034040 tx=0x7f1c38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38040020 con 0x7f1c2c005c60 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c38003940 con 0x7f1c2c005c60 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c440ceab0 con 0x7f1c2c005c60 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.812+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38036040 con 0x7f1c2c005c60 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.813+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c440cf070 con 0x7f1c2c005c60 2024-04-26T20:00:32.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.814+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c440ce540 con 0x7f1c2c005c60 2024-04-26T20:00:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.815+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c38043020 con 0x7f1c2c005c60 2024-04-26T20:00:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.815+0000 7f1c40c00640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 0x7f1c2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.816+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f1c380abd40 con 0x7f1c2c005c60 2024-04-26T20:00:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.816+0000 7f1c43e00640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 0x7f1c2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.816+0000 7f1c43e00640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 0x7f1c2406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c44154360 tx=0x7f1c30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:32.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:32.820+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c3804a070 con 0x7f1c2c005c60 2024-04-26T20:00:33.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.021+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c440693c0 con 0x7f1c2c005c60 2024-04-26T20:00:33.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.022+0000 7f1c40c00640 1 -- 172.21.15.88:0/3755046103 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1c3807a040 con 0x7f1c2c005c60 2024-04-26T20:00:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:33.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.024+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 msgr2=0x7f1c2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.024+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 0x7f1c2406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c44154360 tx=0x7f1c30009040 comp rx=0 tx=0).stop 2024-04-26T20:00:33.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.024+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 msgr2=0x7f1c4414cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.024+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c38034040 tx=0x7f1c38004730 comp rx=0 tx=0).stop 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 shutdown_connections 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c4414aa60 0x7f1c44072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c2406c920 0x7f1c2406ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c440ce160 0x7f1c440cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 --2- 172.21.15.88:0/3755046103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c005c60 0x7f1c4414cde0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 >> 172.21.15.88:0/3755046103 conn(0x7f1c440b2900 msgr2=0x7f1c4415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 shutdown_connections 2024-04-26T20:00:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.025+0000 7f1c4a600640 1 -- 172.21.15.88:0/3755046103 wait complete. 2024-04-26T20:00:33.045 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.249+0000 7f499d400640 1 Processor -- start 2024-04-26T20:00:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.249+0000 7f499d400640 1 -- start start 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f499d400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4998156650 0x7f499815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f499d400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f499806c770 0x7f499806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f499d400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499806eea0 con 0x7f4998073120 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f499d400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f49980b4c40 con 0x7f4998156650 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f499d400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f499806f140 con 0x7f499806c770 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f4996800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4998156650 0x7f499815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f4996800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4998156650 0x7f499815cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40422/0 (socket says 172.21.15.88:40422) 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f4996800640 1 -- 172.21.15.88:0/1231365747 learned_addr learned my addr 172.21.15.88:0/1231365747 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.251+0000 7f4995e00640 1 --2- 172.21.15.88:0/1231365747 >> v2:172.21.15.132:3300/0 conn(0x7f499806c770 0x7f499806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.252+0000 7f4996800640 1 -- 172.21.15.88:0/1231365747 >> v2:172.21.15.132:3300/0 conn(0x7f499806c770 msgr2=0x7f499806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.252+0000 7f4996800640 1 --2- 172.21.15.88:0/1231365747 >> v2:172.21.15.132:3300/0 conn(0x7f499806c770 0x7f499806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.252+0000 7f4996800640 1 -- 172.21.15.88:0/1231365747 >> v1:172.21.15.132:6789/0 conn(0x7f4998073120 legacy=0x7f4998073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:33.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.252+0000 7f4996800640 1 -- 172.21.15.88:0/1231365747 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499806f420 con 0x7f4998156650 2024-04-26T20:00:33.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.253+0000 7f4996800640 1 --2- 172.21.15.88:0/1231365747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f499815cde0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49980d1d90 tx=0x7f498802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5478564dff2b9de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.253+0000 7f4995400640 1 -- 172.21.15.88:0/1231365747 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988037070 con 0x7f4998156650 2024-04-26T20:00:33.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.253+0000 7f4995400640 1 -- 172.21.15.88:0/1231365747 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4988032440 con 0x7f4998156650 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.253+0000 7f4995400640 1 -- 172.21.15.88:0/1231365747 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988032810 con 0x7f4998156650 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 -- 172.21.15.88:0/1231365747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 msgr2=0x7f499815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 --2- 172.21.15.88:0/1231365747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f499815cde0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f49980d1d90 tx=0x7f498802a230 comp rx=0 tx=0).stop 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 -- 172.21.15.88:0/1231365747 shutdown_connections 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 --2- 172.21.15.88:0/1231365747 >> v2:172.21.15.132:3300/0 conn(0x7f499806c770 0x7f499806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 --2- 172.21.15.88:0/1231365747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f499815cde0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 -- 172.21.15.88:0/1231365747 >> 172.21.15.88:0/1231365747 conn(0x7f49980b2d50 msgr2=0x7f49980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 -- 172.21.15.88:0/1231365747 shutdown_connections 2024-04-26T20:00:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.254+0000 7f499d400640 1 -- 172.21.15.88:0/1231365747 wait complete. 2024-04-26T20:00:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.256+0000 7f499d400640 1 Processor -- start 2024-04-26T20:00:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.256+0000 7f499d400640 1 -- start start 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f499806c770 0x7f499814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 0x7f49981e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f499806fcc0 con 0x7f4998156650 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f499806f9e0 con 0x7f4998073120 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f499d400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f499815d320 con 0x7f499806c770 2024-04-26T20:00:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.257+0000 7f4997200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40438/0 (socket says 172.21.15.88:40438) 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 -- 172.21.15.88:0/3725984644 learned_addr learned my addr 172.21.15.88:0/3725984644 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4995e00640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 0x7f49981e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4996800640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f499806c770 0x7f499814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 -- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f499806c770 msgr2=0x7f499814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f499806c770 0x7f499814ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 -- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 msgr2=0x7f49981e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 0x7f49981e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 -- 172.21.15.88:0/3725984644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49981e7400 con 0x7f4998156650 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4995e00640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 0x7f49981e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.258+0000 7f4997200640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f499000c7b0 tx=0x7f499000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.259+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990004b60 con 0x7f4998156650 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.259+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4990004d00 con 0x7f4998156650 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.259+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499000aa90 con 0x7f4998156650 2024-04-26T20:00:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.259+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49981e76e0 con 0x7f4998156650 2024-04-26T20:00:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.259+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49980cd520 con 0x7f4998156650 2024-04-26T20:00:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.260+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4998069650 con 0x7f4998156650 2024-04-26T20:00:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.261+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f499000ac30 con 0x7f4998156650 2024-04-26T20:00:33.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.262+0000 7f498ea00640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.262+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f499008bec0 con 0x7f4998156650 2024-04-26T20:00:33.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.262+0000 7f4996800640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.263+0000 7f4996800640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 0x7f497806edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4988034000 tx=0x7f498802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.267+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4990020070 con 0x7f4998156650 2024-04-26T20:00:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.472+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49980cd6f0 con 0x7f4998156650 2024-04-26T20:00:33.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.474+0000 7f498ea00640 1 -- 172.21.15.88:0/3725984644 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f499005a250 con 0x7f4998156650 2024-04-26T20:00:33.475 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 msgr2=0x7f497806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 0x7f497806edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4988034000 tx=0x7f498802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:33.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 msgr2=0x7f49981e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f499000c7b0 tx=0x7f499000cc70 comp rx=0 tx=0).stop 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 shutdown_connections 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4998156650 0x7f49981e6df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4998073120 0x7f49981e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f497806c8f0 0x7f497806edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 --2- 172.21.15.88:0/3725984644 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f499806c770 0x7f499814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.476+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 >> 172.21.15.88:0/3725984644 conn(0x7f49980b2d50 msgr2=0x7f49980b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.477+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 shutdown_connections 2024-04-26T20:00:33.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.477+0000 7f499d400640 1 -- 172.21.15.88:0/3725984644 wait complete. 2024-04-26T20:00:33.498 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:33.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.700+0000 7f1023200640 1 Processor -- start 2024-04-26T20:00:33.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.700+0000 7f1023200640 1 -- start start 2024-04-26T20:00:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1023200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 0x7f10240d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1023200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f10240d12a0 0x7f10240b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1023200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1024160b20 con 0x7f10241467a0 2024-04-26T20:00:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1023200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1024146b80 con 0x7f10240d12a0 2024-04-26T20:00:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1023200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1024161730 con 0x7f10240b5ca0 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1021e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f10241467a0 0x7f102415dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56954/0 (socket says 172.21.15.88:56954) 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1021e00640 1 -- 172.21.15.88:0/2895813456 learned_addr learned my addr 172.21.15.88:0/2895813456 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.702+0000 7f1022800640 1 --2- 172.21.15.88:0/2895813456 >> v2:172.21.15.88:3300/0 conn(0x7f10240d12a0 0x7f10240b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1022800640 1 -- 172.21.15.88:0/2895813456 >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 msgr2=0x7f10240d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1021400640 1 --2- 172.21.15.88:0/2895813456 >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 0x7f10240d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1022800640 1 --2- 172.21.15.88:0/2895813456 >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 0x7f10240d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1022800640 1 -- 172.21.15.88:0/2895813456 >> v1:172.21.15.88:6789/0 conn(0x7f10241467a0 legacy=0x7f102415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1022800640 1 -- 172.21.15.88:0/2895813456 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1024160e00 con 0x7f10240d12a0 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1021400640 1 --2- 172.21.15.88:0/2895813456 >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 0x7f10240d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.703+0000 7f1020a00640 1 -- 172.21.15.88:0/2895813456 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847231147 0 0) 0x7f1024160b20 con 0x7f10241467a0 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.704+0000 7f1022800640 1 --2- 172.21.15.88:0/2895813456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10240d12a0 0x7f10240b40f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1024160950 tx=0x7f101c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4169ca160afef60e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.704+0000 7f1020a00640 1 -- 172.21.15.88:0/2895813456 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c007740 con 0x7f10240d12a0 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.704+0000 7f1020a00640 1 -- 172.21.15.88:0/2895813456 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c0078e0 con 0x7f10240d12a0 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.704+0000 7f1020a00640 1 -- 172.21.15.88:0/2895813456 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c007cb0 con 0x7f10240d12a0 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 -- 172.21.15.88:0/2895813456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10240d12a0 msgr2=0x7f10240b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 --2- 172.21.15.88:0/2895813456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10240d12a0 0x7f10240b40f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1024160950 tx=0x7f101c02a680 comp rx=0 tx=0).stop 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 -- 172.21.15.88:0/2895813456 shutdown_connections 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 --2- 172.21.15.88:0/2895813456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10240d12a0 0x7f10240b40f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 --2- 172.21.15.88:0/2895813456 >> v2:172.21.15.132:3300/0 conn(0x7f10240b5ca0 0x7f10240d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 -- 172.21.15.88:0/2895813456 >> 172.21.15.88:0/2895813456 conn(0x7f102406c530 msgr2=0x7f102406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 -- 172.21.15.88:0/2895813456 shutdown_connections 2024-04-26T20:00:33.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.705+0000 7f1023200640 1 -- 172.21.15.88:0/2895813456 wait complete. 2024-04-26T20:00:33.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.706+0000 7f1023200640 1 Processor -- start 2024-04-26T20:00:33.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 -- start start 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 0x7f10241f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 0x7f10241f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10241613c0 con 0x7f10241467a0 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f10240d16d0 con 0x7f10240b5ca0 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.707+0000 7f1023200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f10240cf100 con 0x7f10240d12a0 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40454/0 (socket says 172.21.15.88:40454) 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 -- 172.21.15.88:0/1844942217 learned_addr learned my addr 172.21.15.88:0/1844942217 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1021e00640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 0x7f10241f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1021400640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 0x7f10241f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 -- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 msgr2=0x7f10241f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 0x7f10241f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 -- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 msgr2=0x7f10241f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 0x7f10241f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1022800640 1 -- 172.21.15.88:0/1844942217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10241f6740 con 0x7f10241467a0 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1021e00640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 0x7f10241f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.708+0000 7f1021400640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 0x7f10241f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f1022800640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f101c009e80 tx=0x7f101c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c040070 con 0x7f10241467a0 2024-04-26T20:00:33.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c031c40 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c003b90 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10241f6a20 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.709+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10240c5e10 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.710+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10240c6010 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.711+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101c039040 con 0x7f10241467a0 2024-04-26T20:00:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.712+0000 7f101a000640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.712+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f101c0acf00 con 0x7f10241467a0 2024-04-26T20:00:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.712+0000 7f1021e00640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:33.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.713+0000 7f1021e00640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1014000f60 tx=0x7f1014009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:33.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.717+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10240c6010 con 0x7f10241467a0 2024-04-26T20:00:33.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.915+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f102415e460 con 0x7f10241467a0 2024-04-26T20:00:33.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.916+0000 7f101a000640 1 -- 172.21.15.88:0/1844942217 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f101c07b290 con 0x7f10241467a0 2024-04-26T20:00:33.918 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 msgr2=0x7f100406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1014000f60 tx=0x7f1014009040 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 msgr2=0x7f102406de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f101c009e80 tx=0x7f101c004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 shutdown_connections 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10241467a0 0x7f102406de40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f10240d12a0 0x7f10241f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 --2- 172.21.15.88:0/1844942217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10240b5ca0 0x7f10241f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 >> 172.21.15.88:0/1844942217 conn(0x7f102406c530 msgr2=0x7f10240b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 shutdown_connections 2024-04-26T20:00:33.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:33.919+0000 7f1023200640 1 -- 172.21.15.88:0/1844942217 wait complete. 2024-04-26T20:00:33.940 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:34.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.133+0000 7f211d200640 1 Processor -- start 2024-04-26T20:00:34.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.134+0000 7f211d200640 1 -- start start 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.135+0000 7f211d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f211815cdf0 0x7f21180add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.135+0000 7f211d200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f211806ce80 0x7f21180b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.135+0000 7f211d200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21180af9a0 con 0x7f21180cd650 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.135+0000 7f211d200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f21180ae2c0 con 0x7f211806ce80 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.135+0000 7f211d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f21180b05b0 con 0x7f211815cdf0 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2116800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f211806ce80 0x7f21180b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2115400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f21180cd650 0x7f211806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56968/0 (socket says 172.21.15.88:56968) 2024-04-26T20:00:34.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2115400640 1 -- 172.21.15.88:0/3436000562 learned_addr learned my addr 172.21.15.88:0/3436000562 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2115e00640 1 --2- 172.21.15.88:0/3436000562 >> v2:172.21.15.132:3300/0 conn(0x7f211815cdf0 0x7f21180add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2116800640 1 -- 172.21.15.88:0/3436000562 >> v2:172.21.15.132:3300/0 conn(0x7f211815cdf0 msgr2=0x7f21180add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2116800640 1 --2- 172.21.15.88:0/3436000562 >> v2:172.21.15.132:3300/0 conn(0x7f211815cdf0 0x7f21180add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2116800640 1 -- 172.21.15.88:0/3436000562 >> v1:172.21.15.88:6789/0 conn(0x7f21180cd650 legacy=0x7f211806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2116800640 1 -- 172.21.15.88:0/3436000562 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21180afc80 con 0x7f211806ce80 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.136+0000 7f2114a00640 1 -- 172.21.15.88:0/3436000562 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036057345 0 0) 0x7f21180af9a0 con 0x7f21180cd650 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.137+0000 7f2116800640 1 --2- 172.21.15.88:0/3436000562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180b3d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21180b3950 tx=0x7f211002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657b7a5f857bfcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f2114a00640 1 -- 172.21.15.88:0/3436000562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110007740 con 0x7f211806ce80 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f2114a00640 1 -- 172.21.15.88:0/3436000562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21100078e0 con 0x7f211806ce80 2024-04-26T20:00:34.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f2114a00640 1 -- 172.21.15.88:0/3436000562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110007cb0 con 0x7f211806ce80 2024-04-26T20:00:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 -- 172.21.15.88:0/3436000562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 msgr2=0x7f21180b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 --2- 172.21.15.88:0/3436000562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180b3d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21180b3950 tx=0x7f211002a680 comp rx=0 tx=0).stop 2024-04-26T20:00:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 -- 172.21.15.88:0/3436000562 shutdown_connections 2024-04-26T20:00:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 --2- 172.21.15.88:0/3436000562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180b3d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 --2- 172.21.15.88:0/3436000562 >> v2:172.21.15.132:3300/0 conn(0x7f211815cdf0 0x7f21180add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 -- 172.21.15.88:0/3436000562 >> 172.21.15.88:0/3436000562 conn(0x7f21180b2d50 msgr2=0x7f21180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 -- 172.21.15.88:0/3436000562 shutdown_connections 2024-04-26T20:00:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.138+0000 7f211d200640 1 -- 172.21.15.88:0/3436000562 wait complete. 2024-04-26T20:00:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.140+0000 7f211d200640 1 Processor -- start 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.140+0000 7f211d200640 1 -- start start 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21180cd650 0x7f21181562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f211815cdf0 0x7f211806f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f211815d1d0 con 0x7f211806ce80 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21180b0240 con 0x7f211815cdf0 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f211d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f211815d350 con 0x7f21180cd650 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f2115e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f2115e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40464/0 (socket says 172.21.15.88:40464) 2024-04-26T20:00:34.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.141+0000 7f2115e00640 1 -- 172.21.15.88:0/3629041111 learned_addr learned my addr 172.21.15.88:0/3629041111 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 -- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21180cd650 msgr2=0x7f21181562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21180cd650 0x7f21181562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 -- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f211815cdf0 msgr2=0x7f211806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2116800640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f211815cdf0 0x7f211806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f211815cdf0 0x7f211806f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 -- 172.21.15.88:0/3629041111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211815e440 con 0x7f211806ce80 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f2115e00640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f210800d5f0 tx=0x7f210800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080125f0 con 0x7f211806ce80 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2108019070 con 0x7f211806ce80 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.142+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21180ca4f0 con 0x7f211806ce80 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.143+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21080129d0 con 0x7f211806ce80 2024-04-26T20:00:34.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.143+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21180caa70 con 0x7f211806ce80 2024-04-26T20:00:34.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.144+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2118069d00 con 0x7f211806ce80 2024-04-26T20:00:34.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.145+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2108012b70 con 0x7f211806ce80 2024-04-26T20:00:34.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.145+0000 7f210e000640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 0x7f20f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.146+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f210808bf30 con 0x7f211806ce80 2024-04-26T20:00:34.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.146+0000 7f2115400640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 0x7f20f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.146+0000 7f2115400640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 0x7f20f406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2100007fd0 tx=0x7f2100007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.150+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2118069d00 con 0x7f211806ce80 2024-04-26T20:00:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.369+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2118156f40 con 0x7f211806ce80 2024-04-26T20:00:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.370+0000 7f210e000640 1 -- 172.21.15.88:0/3629041111 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f211815e440 con 0x7f211806ce80 2024-04-26T20:00:34.371 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:34.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.372+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 msgr2=0x7f20f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.372+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 0x7f20f406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2100007fd0 tx=0x7f2100007920 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.372+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 msgr2=0x7f21180c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.372+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f210800d5f0 tx=0x7f210800dab0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 shutdown_connections 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f211815cdf0 0x7f211806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f20f406c8f0 0x7f20f406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21180cd650 0x7f21181562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 --2- 172.21.15.88:0/3629041111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f211806ce80 0x7f21180c9fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 >> 172.21.15.88:0/3629041111 conn(0x7f21180b2d50 msgr2=0x7f211806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:34.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 shutdown_connections 2024-04-26T20:00:34.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.373+0000 7f211d200640 1 -- 172.21.15.88:0/3629041111 wait complete. 2024-04-26T20:00:34.393 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-04-26T20:00:34.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:34.393 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:00:34.478 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:34.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:00:34.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:34.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:00:34.540 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:34.541 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:34.600 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:34.601 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:34.653 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.857+0000 7f68ef400640 1 Processor -- start 2024-04-26T20:00:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.857+0000 7f68ef400640 1 -- start start 2024-04-26T20:00:34.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68ef400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f68e806ce80 0x7f68e80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68ef400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e80af9a0 con 0x7f68e815cdf0 2024-04-26T20:00:34.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68ef400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68e80afc80 con 0x7f68e80cd650 2024-04-26T20:00:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68ef400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f68e80ae2c0 con 0x7f68e806ce80 2024-04-26T20:00:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68eca00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f68e815cdf0 0x7f68e806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56984/0 (socket says 172.21.15.88:56984) 2024-04-26T20:00:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.859+0000 7f68eca00640 1 -- 172.21.15.88:0/3046204179 learned_addr learned my addr 172.21.15.88:0/3046204179 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:34.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546988189 0 0) 0x7f68e80af9a0 con 0x7f68e815cdf0 2024-04-26T20:00:34.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68d0003520 con 0x7f68e815cdf0 2024-04-26T20:00:34.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7e00640 1 --2- 172.21.15.88:0/3046204179 >> v2:172.21.15.132:3300/0 conn(0x7f68e806ce80 0x7f68e80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1786561707 0 0) 0x7f68d0003520 con 0x7f68e815cdf0 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 >> v2:172.21.15.132:3300/0 conn(0x7f68e806ce80 msgr2=0x7f68e80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 --2- 172.21.15.88:0/3046204179 >> v2:172.21.15.132:3300/0 conn(0x7f68e806ce80 0x7f68e80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 >> v1:172.21.15.132:6789/0 conn(0x7f68e80cd650 legacy=0x7f68e80b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e80aff60 con 0x7f68e815cdf0 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f68dc001670 con 0x7f68e815cdf0 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 >> v1:172.21.15.88:6789/0 conn(0x7f68e815cdf0 legacy=0x7f68e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68d0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68d000c0c0 con 0x7f68d00039b0 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f68d000c270 con 0x7f68d00051c0 2024-04-26T20:00:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f68d000c470 con 0x7f68d0004800 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.860+0000 7f68eca00640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68d0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68ed400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68e7e00640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68eca00640 1 -- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68d0004800 msgr2=0x7f68d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68eca00640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68d0004800 0x7f68d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68eca00640 1 -- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68d00051c0 msgr2=0x7f68d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68eca00640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68d00051c0 0x7f68d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68eca00640 1 -- 172.21.15.88:0/3046204179 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d0003520 con 0x7f68d00039b0 2024-04-26T20:00:34.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.861+0000 7f68ed400640 1 -- 172.21.15.88:0/3046204179 reap_dead start 2024-04-26T20:00:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.862+0000 7f68eca00640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68d0003db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c210 tx=0x7f68d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8573515915f86328 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.862+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc007d10 con 0x7f68d00039b0 2024-04-26T20:00:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.862+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc0058d0 con 0x7f68d00039b0 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.862+0000 7f68e7400640 1 -- 172.21.15.88:0/3046204179 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc005d40 con 0x7f68d00039b0 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 -- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 msgr2=0x7f68d0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68d0003db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c210 tx=0x7f68d0005830 comp rx=0 tx=0).stop 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 -- 172.21.15.88:0/3046204179 shutdown_connections 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 --2- 172.21.15.88:0/3046204179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68d0003db0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 -- 172.21.15.88:0/3046204179 >> 172.21.15.88:0/3046204179 conn(0x7f68e80b2d50 msgr2=0x7f68e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 -- 172.21.15.88:0/3046204179 shutdown_connections 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.863+0000 7f68ef400640 1 -- 172.21.15.88:0/3046204179 wait complete. 2024-04-26T20:00:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.865+0000 7f68ef400640 1 Processor -- start 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.865+0000 7f68ef400640 1 -- start start 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68e80c9fb0 0x7f68e80ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e80ceb40 0x7f68e80cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68e815d1d0 con 0x7f68d00039b0 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f68e80b05b0 con 0x7f68e80ceb40 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.866+0000 7f68ef400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f68e815d350 con 0x7f68e80c9fb0 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40490/0 (socket says 172.21.15.88:40490) 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 -- 172.21.15.88:0/215412141 learned_addr learned my addr 172.21.15.88:0/215412141 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:34.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68e7e00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68e80c9fb0 0x7f68e80ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68ed400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e80ceb40 0x7f68e80cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 -- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68e80c9fb0 msgr2=0x7f68e80ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68e80c9fb0 0x7f68e80ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 -- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e80ceb40 msgr2=0x7f68e80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e80ceb40 0x7f68e80cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.867+0000 7f68eca00640 1 -- 172.21.15.88:0/215412141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e81f4510 con 0x7f68d00039b0 2024-04-26T20:00:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68eca00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c210 tx=0x7f68e80b0130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc041020 con 0x7f68d00039b0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc007070 con 0x7f68d00039b0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e814d5b0 con 0x7f68d00039b0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc038040 con 0x7f68d00039b0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.868+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e814db70 con 0x7f68d00039b0 2024-04-26T20:00:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.869+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e8069540 con 0x7f68d00039b0 2024-04-26T20:00:34.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.870+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68dc044020 con 0x7f68d00039b0 2024-04-26T20:00:34.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.871+0000 7f68e4c00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.871+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f68dc0abc80 con 0x7f68d00039b0 2024-04-26T20:00:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.871+0000 7f68e7e00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.872+0000 7f68e7e00640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 0x7f68c806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68e8150ef0 tx=0x7f68d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:34.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:34.875+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68dc04b030 con 0x7f68d00039b0 2024-04-26T20:00:35.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.078+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68e80cf500 con 0x7f68d00039b0 2024-04-26T20:00:35.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.080+0000 7f68e4c00640 1 -- 172.21.15.88:0/215412141 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f68dc07a020 con 0x7f68d00039b0 2024-04-26T20:00:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:35.082 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 msgr2=0x7f68c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 0x7f68c806ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68e8150ef0 tx=0x7f68d4008040 comp rx=0 tx=0).stop 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 msgr2=0x7f68e81f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c210 tx=0x7f68e80b0130 comp rx=0 tx=0).stop 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 shutdown_connections 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e80ceb40 0x7f68e80cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68e80c9fb0 0x7f68e80ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 --2- 172.21.15.88:0/215412141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68d00039b0 0x7f68e81f3fd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.083+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 >> 172.21.15.88:0/215412141 conn(0x7f68e80b2d50 msgr2=0x7f68e806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.084+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 shutdown_connections 2024-04-26T20:00:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.084+0000 7f68ef400640 1 -- 172.21.15.88:0/215412141 wait complete. 2024-04-26T20:00:35.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:35.104 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.309+0000 7f258e800640 1 Processor -- start 2024-04-26T20:00:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.310+0000 7f258e800640 1 -- start start 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2588152430 0x7f25880b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2588152810 0x7f25880b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588072320 0x7f25880727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f258806e130 con 0x7f2588152810 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f2588073090 con 0x7f2588152430 2024-04-26T20:00:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.311+0000 7f258e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2588073370 con 0x7f2588072320 2024-04-26T20:00:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2588152810 0x7f25880b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2588152810 0x7f25880b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40496/0 (socket says 172.21.15.88:40496) 2024-04-26T20:00:35.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 -- 172.21.15.88:0/2679900516 learned_addr learned my addr 172.21.15.88:0/2679900516 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f2587e00640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588072320 0x7f25880727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f2587400640 1 --2- 172.21.15.88:0/2679900516 >> v2:172.21.15.132:3300/0 conn(0x7f2588152430 0x7f25880b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 -- 172.21.15.88:0/2679900516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588072320 msgr2=0x7f25880727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588072320 0x7f25880727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 -- 172.21.15.88:0/2679900516 >> v2:172.21.15.132:3300/0 conn(0x7f2588152430 msgr2=0x7f25880b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 --2- 172.21.15.88:0/2679900516 >> v2:172.21.15.132:3300/0 conn(0x7f2588152430 0x7f25880b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.312+0000 7f257fe00640 1 -- 172.21.15.88:0/2679900516 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2588073650 con 0x7f2588152810 2024-04-26T20:00:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.313+0000 7f257fe00640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152810 0x7f25880b4a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f258806d2a0 tx=0x7f257002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c0355f64e213ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f2586a00640 1 -- 172.21.15.88:0/2679900516 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570037070 con 0x7f2588152810 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f2586a00640 1 -- 172.21.15.88:0/2679900516 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2570032440 con 0x7f2588152810 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f2586a00640 1 -- 172.21.15.88:0/2679900516 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570032810 con 0x7f2588152810 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 -- 172.21.15.88:0/2679900516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152810 msgr2=0x7f25880b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152810 0x7f25880b4a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f258806d2a0 tx=0x7f257002a270 comp rx=0 tx=0).stop 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 -- 172.21.15.88:0/2679900516 shutdown_connections 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588072320 0x7f25880727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 --2- 172.21.15.88:0/2679900516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152810 0x7f25880b4a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 --2- 172.21.15.88:0/2679900516 >> v2:172.21.15.132:3300/0 conn(0x7f2588152430 0x7f25880b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 -- 172.21.15.88:0/2679900516 >> 172.21.15.88:0/2679900516 conn(0x7f25880b2d50 msgr2=0x7f25880b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.314+0000 7f258e800640 1 -- 172.21.15.88:0/2679900516 shutdown_connections 2024-04-26T20:00:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.315+0000 7f258e800640 1 -- 172.21.15.88:0/2679900516 wait complete. 2024-04-26T20:00:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.316+0000 7f258e800640 1 Processor -- start 2024-04-26T20:00:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.316+0000 7f258e800640 1 -- start start 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 0x7f25880af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588152810 0x7f2588150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2588072e70 con 0x7f2588152430 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2588072cf0 con 0x7f2588072320 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.317+0000 7f258e800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f258806c770 con 0x7f2588152810 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40510/0 (socket says 172.21.15.88:40510) 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 -- 172.21.15.88:0/1092502173 learned_addr learned my addr 172.21.15.88:0/1092502173 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f2587400640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 0x7f25880af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f2587e00640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588152810 0x7f2588150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 -- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588152810 msgr2=0x7f2588150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588152810 0x7f2588150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 -- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 msgr2=0x7f25880af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 0x7f25880af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f257fe00640 1 -- 172.21.15.88:0/1092502173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880b22b0 con 0x7f2588152430 2024-04-26T20:00:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.318+0000 7f2587400640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 0x7f25880af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f257fe00640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257002a2b0 tx=0x7f25700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570041020 con 0x7f2588152430 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25700306c0 con 0x7f2588152430 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570037040 con 0x7f2588152430 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25880b09d0 con 0x7f2588152430 2024-04-26T20:00:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.319+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25880b0f90 con 0x7f2588152430 2024-04-26T20:00:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.320+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2588069650 con 0x7f2588152430 2024-04-26T20:00:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.321+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2570044020 con 0x7f2588152430 2024-04-26T20:00:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.322+0000 7f257f400640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 0x7f256406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.322+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f25700ac470 con 0x7f2588152430 2024-04-26T20:00:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.322+0000 7f2587400640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 0x7f256406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.323+0000 7f2587400640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 0x7f256406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2578006fd0 tx=0x7f2578008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.326+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f257004b030 con 0x7f2588152430 2024-04-26T20:00:35.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.527+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f258815b570 con 0x7f2588152430 2024-04-26T20:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.528+0000 7f257f400640 1 -- 172.21.15.88:0/1092502173 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f257007a800 con 0x7f2588152430 2024-04-26T20:00:35.530 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.530+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 msgr2=0x7f256406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 0x7f256406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2578006fd0 tx=0x7f2578008040 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 msgr2=0x7f25880b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257002a2b0 tx=0x7f25700046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 shutdown_connections 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f256406c8f0 0x7f256406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2588152810 0x7f2588150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2588152430 0x7f25880b14b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 --2- 172.21.15.88:0/1092502173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2588072320 0x7f25880af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 >> 172.21.15.88:0/1092502173 conn(0x7f25880b2d50 msgr2=0x7f25880b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 shutdown_connections 2024-04-26T20:00:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.531+0000 7f258e800640 1 -- 172.21.15.88:0/1092502173 wait complete. 2024-04-26T20:00:35.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.756+0000 7fc709200640 1 Processor -- start 2024-04-26T20:00:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.756+0000 7fc709200640 1 -- start start 2024-04-26T20:00:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc709200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc704160d60 0x7fc7040b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc709200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc704146280 0x7fc7041466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc709200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7040b4d20 con 0x7fc704161140 2024-04-26T20:00:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc709200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc70406e5f0 con 0x7fc704160d60 2024-04-26T20:00:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc709200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc704146c30 con 0x7fc704146280 2024-04-26T20:00:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc703400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc704160d60 0x7fc7040b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc703400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc704160d60 0x7fc7040b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40512/0 (socket says 172.21.15.88:40512) 2024-04-26T20:00:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.758+0000 7fc703400640 1 -- 172.21.15.88:0/3227968813 learned_addr learned my addr 172.21.15.88:0/3227968813 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.759+0000 7fc703e00640 1 --2- 172.21.15.88:0/3227968813 >> v2:172.21.15.132:3300/0 conn(0x7fc704146280 0x7fc7041466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.759+0000 7fc703400640 1 -- 172.21.15.88:0/3227968813 >> v2:172.21.15.132:3300/0 conn(0x7fc704146280 msgr2=0x7fc7041466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.759+0000 7fc703400640 1 --2- 172.21.15.88:0/3227968813 >> v2:172.21.15.132:3300/0 conn(0x7fc704146280 0x7fc7041466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.759+0000 7fc703400640 1 -- 172.21.15.88:0/3227968813 >> v1:172.21.15.132:6789/0 conn(0x7fc704161140 legacy=0x7fc70406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.759+0000 7fc703400640 1 -- 172.21.15.88:0/3227968813 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7040b5000 con 0x7fc704160d60 2024-04-26T20:00:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.760+0000 7fc703400640 1 --2- 172.21.15.88:0/3227968813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc7040b5b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009870 tx=0x7fc6f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9b44e243229377 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc702000640 1 -- 172.21.15.88:0/3227968813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4037070 con 0x7fc704160d60 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc702000640 1 -- 172.21.15.88:0/3227968813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f4032440 con 0x7fc704160d60 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc702000640 1 -- 172.21.15.88:0/3227968813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4032810 con 0x7fc704160d60 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 -- 172.21.15.88:0/3227968813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 msgr2=0x7fc7040b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 --2- 172.21.15.88:0/3227968813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc7040b5b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009870 tx=0x7fc6f402a270 comp rx=0 tx=0).stop 2024-04-26T20:00:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 -- 172.21.15.88:0/3227968813 shutdown_connections 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 --2- 172.21.15.88:0/3227968813 >> v2:172.21.15.132:3300/0 conn(0x7fc704146280 0x7fc7041466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 --2- 172.21.15.88:0/3227968813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc7040b5b30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.761+0000 7fc709200640 1 -- 172.21.15.88:0/3227968813 >> 172.21.15.88:0/3227968813 conn(0x7fc7040b2900 msgr2=0x7fc7040b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.762+0000 7fc709200640 1 -- 172.21.15.88:0/3227968813 shutdown_connections 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.762+0000 7fc709200640 1 -- 172.21.15.88:0/3227968813 wait complete. 2024-04-26T20:00:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.763+0000 7fc709200640 1 Processor -- start 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.763+0000 7fc709200640 1 -- start start 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc704146280 0x7fc70413c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 0x7fc70413de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7040cf6f0 con 0x7fc704160d60 2024-04-26T20:00:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc7040cf410 con 0x7fc704161140 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc709200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc7040b6070 con 0x7fc704146280 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.764+0000 7fc702a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40522/0 (socket says 172.21.15.88:40522) 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 -- 172.21.15.88:0/1488521216 learned_addr learned my addr 172.21.15.88:0/1488521216 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc703e00640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 0x7fc70413de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 -- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc704146280 msgr2=0x7fc70413c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc703400640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc704146280 0x7fc70413c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc704146280 0x7fc70413c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 -- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 msgr2=0x7fc70413de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 0x7fc70413de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 -- 172.21.15.88:0/1488521216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70415ccb0 con 0x7fc704160d60 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc703e00640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 0x7fc70413de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:35.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc702a00640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000c430 tx=0x7fc6f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0011790 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.765+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f0017070 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.766+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0011bb0 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.766+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70415cf90 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.766+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70415d550 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.767+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7040693c0 con 0x7fc704160d60 2024-04-26T20:00:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.768+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6f001c460 con 0x7fc704160d60 2024-04-26T20:00:35.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.769+0000 7fc6fb400640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 0x7fc6e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:35.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.769+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fc6f008bb00 con 0x7fc704160d60 2024-04-26T20:00:35.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.769+0000 7fc703400640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 0x7fc6e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:35.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.769+0000 7fc703400640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 0x7fc6e406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4034000 tx=0x7fc6f402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:35.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.773+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6f0059e00 con 0x7fc704160d60 2024-04-26T20:00:35.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.973+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7041606b0 con 0x7fc704160d60 2024-04-26T20:00:35.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.974+0000 7fc6fb400640 1 -- 172.21.15.88:0/1488521216 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fc6f008be20 con 0x7fc704160d60 2024-04-26T20:00:35.977 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:35.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 msgr2=0x7fc6e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 0x7fc6e406ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4034000 tx=0x7fc6f402e000 comp rx=0 tx=0).stop 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 msgr2=0x7fc70413d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000c430 tx=0x7fc6f000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 shutdown_connections 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc704161140 0x7fc70413de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc704160d60 0x7fc70413d8e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6e406c920 0x7fc6e406ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 --2- 172.21.15.88:0/1488521216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc704146280 0x7fc70413c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 >> 172.21.15.88:0/1488521216 conn(0x7fc7040b2900 msgr2=0x7fc70415ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 shutdown_connections 2024-04-26T20:00:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:35.977+0000 7fc709200640 1 -- 172.21.15.88:0/1488521216 wait complete. 2024-04-26T20:00:35.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.202+0000 7fdbade00640 1 Processor -- start 2024-04-26T20:00:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.202+0000 7fdbade00640 1 -- start start 2024-04-26T20:00:36.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.203+0000 7fdbade00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdba80d1a90 0x7fdba806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.203+0000 7fdbade00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdba80d0ca0 0x7fdba80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.203+0000 7fdbade00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba80cee90 con 0x7fdba806ef60 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdbade00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdba8070e10 con 0x7fdba80d1a90 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdbade00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fdba806d6e0 con 0x7fdba80d0ca0 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdba80d1a90 0x7fdba806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdba80d1a90 0x7fdba806d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40528/0 (socket says 172.21.15.88:40528) 2024-04-26T20:00:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 -- 172.21.15.88:0/2583084047 learned_addr learned my addr 172.21.15.88:0/2583084047 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdbad400640 1 --2- 172.21.15.88:0/2583084047 >> v2:172.21.15.132:3300/0 conn(0x7fdba80d0ca0 0x7fdba80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 -- 172.21.15.88:0/2583084047 >> v2:172.21.15.132:3300/0 conn(0x7fdba80d0ca0 msgr2=0x7fdba80712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 --2- 172.21.15.88:0/2583084047 >> v2:172.21.15.132:3300/0 conn(0x7fdba80d0ca0 0x7fdba80712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 -- 172.21.15.88:0/2583084047 >> v1:172.21.15.132:6789/0 conn(0x7fdba806ef60 legacy=0x7fdba80d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.204+0000 7fdba7e00640 1 -- 172.21.15.88:0/2583084047 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba8070200 con 0x7fdba80d1a90 2024-04-26T20:00:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.205+0000 7fdba7e00640 1 --2- 172.21.15.88:0/2583084047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdba8073570 tx=0x7fdb9002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead5a9e737eb1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdba7400640 1 -- 172.21.15.88:0/2583084047 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90037070 con 0x7fdba80d1a90 2024-04-26T20:00:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdba7400640 1 -- 172.21.15.88:0/2583084047 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90032440 con 0x7fdba80d1a90 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdba7400640 1 -- 172.21.15.88:0/2583084047 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90032810 con 0x7fdba80d1a90 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 -- 172.21.15.88:0/2583084047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 msgr2=0x7fdba806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 --2- 172.21.15.88:0/2583084047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdba8073570 tx=0x7fdb9002a2c0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 -- 172.21.15.88:0/2583084047 shutdown_connections 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 --2- 172.21.15.88:0/2583084047 >> v2:172.21.15.132:3300/0 conn(0x7fdba80d0ca0 0x7fdba80712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 --2- 172.21.15.88:0/2583084047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.206+0000 7fdbade00640 1 -- 172.21.15.88:0/2583084047 >> 172.21.15.88:0/2583084047 conn(0x7fdba80d2e30 msgr2=0x7fdba80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.207+0000 7fdbade00640 1 -- 172.21.15.88:0/2583084047 shutdown_connections 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.207+0000 7fdbade00640 1 -- 172.21.15.88:0/2583084047 wait complete. 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.208+0000 7fdbade00640 1 Processor -- start 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.208+0000 7fdbade00640 1 -- start start 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 0x7fdba80c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba80d0ca0 0x7fdba81f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdba8070aa0 con 0x7fdba80d1a90 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdba80707c0 con 0x7fdba80d0ca0 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.209+0000 7fdbade00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdba806f340 con 0x7fdba806ef60 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40532/0 (socket says 172.21.15.88:40532) 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 -- 172.21.15.88:0/1572231448 learned_addr learned my addr 172.21.15.88:0/1572231448 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdba7e00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba80d0ca0 0x7fdba81f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbaca00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 0x7fdba80c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 -- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 msgr2=0x7fdba80c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 0x7fdba80c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 -- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba80d0ca0 msgr2=0x7fdba81f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba80d0ca0 0x7fdba81f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbad400640 1 -- 172.21.15.88:0/1572231448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba80c6a10 con 0x7fdba80d1a90 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.210+0000 7fdbaca00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 0x7fdba80c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdbad400640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c7f0 tx=0x7fdba000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0015070 con 0x7fdba80d1a90 2024-04-26T20:00:36.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba0004220 con 0x7fdba80d1a90 2024-04-26T20:00:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba81f5b30 con 0x7fdba80d1a90 2024-04-26T20:00:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0007620 con 0x7fdba80d1a90 2024-04-26T20:00:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.211+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba80c8590 con 0x7fdba80d1a90 2024-04-26T20:00:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.212+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba81f6050 con 0x7fdba80d1a90 2024-04-26T20:00:36.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.213+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdba00079d0 con 0x7fdba80d1a90 2024-04-26T20:00:36.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.214+0000 7fdba4c00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.214+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fdba008bd40 con 0x7fdba80d1a90 2024-04-26T20:00:36.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.214+0000 7fdbaca00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.215+0000 7fdbaca00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 0x7fdb8406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb98007eb0 tx=0x7fdb980073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.218+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdba81f6050 con 0x7fdba80d1a90 2024-04-26T20:00:36.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.418+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdba8069ad0 con 0x7fdba80d1a90 2024-04-26T20:00:36.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.419+0000 7fdba4c00640 1 -- 172.21.15.88:0/1572231448 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fdba005a040 con 0x7fdba80d1a90 2024-04-26T20:00:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:36.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 msgr2=0x7fdb8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 0x7fdb8406edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb98007eb0 tx=0x7fdb980073c0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 msgr2=0x7fdba806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c7f0 tx=0x7fdba000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 shutdown_connections 2024-04-26T20:00:36.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdba80d1a90 0x7fdba806d890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdba80d0ca0 0x7fdba81f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb8406c8f0 0x7fdb8406edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 --2- 172.21.15.88:0/1572231448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdba806ef60 0x7fdba80c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 >> 172.21.15.88:0/1572231448 conn(0x7fdba80d2e30 msgr2=0x7fdba80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 shutdown_connections 2024-04-26T20:00:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.422+0000 7fdbade00640 1 -- 172.21.15.88:0/1572231448 wait complete. 2024-04-26T20:00:36.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:36.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.637+0000 7fb9d8a00640 1 Processor -- start 2024-04-26T20:00:36.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.637+0000 7fb9d8a00640 1 -- start start 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 0x7fb9d4071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d8a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d4073a90 con 0x7fb9d406ca60 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d4073d70 con 0x7fb9d406fa50 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d406f5c0 con 0x7fb9d415f150 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d3e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb9d406ca60 0x7fb9d40b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:56996/0 (socket says 172.21.15.88:56996) 2024-04-26T20:00:36.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d3e00640 1 -- 172.21.15.88:0/1861914928 learned_addr learned my addr 172.21.15.88:0/1861914928 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:36.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.639+0000 7fb9d3400640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 0x7fb9d4071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280934685 0 0) 0x7fb9d4073a90 con 0x7fb9d406ca60 2024-04-26T20:00:36.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b8003520 con 0x7fb9d406ca60 2024-04-26T20:00:36.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3883646687 0 0) 0x7fb9b8003520 con 0x7fb9d406ca60 2024-04-26T20:00:36.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 msgr2=0x7fb9d4071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:36.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 0x7fb9d4071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 >> v1:172.21.15.132:6789/0 conn(0x7fb9d406fa50 legacy=0x7fb9d406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d406ef70 con 0x7fb9d406ca60 2024-04-26T20:00:36.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb9cc0019f0 con 0x7fb9d406ca60 2024-04-26T20:00:36.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 >> v1:172.21.15.88:6789/0 conn(0x7fb9d406ca60 legacy=0x7fb9d40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9b8004800 0x7fb9b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 0x7fb9b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d406f5c0 con 0x7fb9b80039e0 2024-04-26T20:00:36.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b800baf0 con 0x7fb9b8004800 2024-04-26T20:00:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b800bcf0 con 0x7fb9d415f150 2024-04-26T20:00:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.640+0000 7fb9d3400640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d3400640 1 -- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 msgr2=0x7fb9b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:36.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d3400640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d415f150 0x7fb9b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d3400640 1 -- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9b8004800 msgr2=0x7fb9b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:36.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d2a00640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9b8004800 0x7fb9b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d3400640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9b8004800 0x7fb9b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d3400640 1 -- 172.21.15.88:0/1861914928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b8003520 con 0x7fb9b80039e0 2024-04-26T20:00:36.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.641+0000 7fb9d2a00640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9b8004800 0x7fb9b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:36.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.642+0000 7fb9d3400640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9b8003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009ac0 tx=0x7fb9b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b578230aeeb12dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.642+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4036070 con 0x7fb9b80039e0 2024-04-26T20:00:36.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.642+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c4031400 con 0x7fb9b80039e0 2024-04-26T20:00:36.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.642+0000 7fb9d2000640 1 -- 172.21.15.88:0/1861914928 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4009690 con 0x7fb9b80039e0 2024-04-26T20:00:36.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 -- 172.21.15.88:0/1861914928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 msgr2=0x7fb9b8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/1861914928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9b8003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009ac0 tx=0x7fb9b8005830 comp rx=0 tx=0).stop 2024-04-26T20:00:36.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d3e00640 1 -- 172.21.15.88:0/1861914928 reap_dead start 2024-04-26T20:00:36.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 -- 172.21.15.88:0/1861914928 shutdown_connections 2024-04-26T20:00:36.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 -- 172.21.15.88:0/1861914928 >> 172.21.15.88:0/1861914928 conn(0x7fb9d40b38a0 msgr2=0x7fb9d40b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:36.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 -- 172.21.15.88:0/1861914928 shutdown_connections 2024-04-26T20:00:36.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.643+0000 7fb9d8a00640 1 -- 172.21.15.88:0/1861914928 wait complete. 2024-04-26T20:00:36.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.645+0000 7fb9d8a00640 1 Processor -- start 2024-04-26T20:00:36.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.645+0000 7fb9d8a00640 1 -- start start 2024-04-26T20:00:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d4156890 0x7fb9d4161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d415cfa0 0x7fb9d415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4071590 con 0x7fb9b80039e0 2024-04-26T20:00:36.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d406feb0 con 0x7fb9d415cfa0 2024-04-26T20:00:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d4071710 con 0x7fb9d4156890 2024-04-26T20:00:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40548/0 (socket says 172.21.15.88:40548) 2024-04-26T20:00:36.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 -- 172.21.15.88:0/3301863829 learned_addr learned my addr 172.21.15.88:0/3301863829 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d2a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d4156890 0x7fb9d4161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 -- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d4156890 msgr2=0x7fb9d4161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d4156890 0x7fb9d4161e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 -- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d415cfa0 msgr2=0x7fb9d415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d415cfa0 0x7fb9d415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.646+0000 7fb9d3400640 1 -- 172.21.15.88:0/3301863829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d415be70 con 0x7fb9b80039e0 2024-04-26T20:00:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9d3400640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009be0 tx=0x7fb9c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4040020 con 0x7fb9b80039e0 2024-04-26T20:00:36.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c40038d0 con 0x7fb9b80039e0 2024-04-26T20:00:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4036040 con 0x7fb9b80039e0 2024-04-26T20:00:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d415c150 con 0x7fb9b80039e0 2024-04-26T20:00:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.647+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d415f780 con 0x7fb9b80039e0 2024-04-26T20:00:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.649+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d4069ad0 con 0x7fb9b80039e0 2024-04-26T20:00:36.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.649+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9c4043020 con 0x7fb9b80039e0 2024-04-26T20:00:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.650+0000 7fb9cb400640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 0x7fb9b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.650+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fb9c40abc20 con 0x7fb9b80039e0 2024-04-26T20:00:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.651+0000 7fb9d2a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 0x7fb9b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.651+0000 7fb9d2a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 0x7fb9b406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0007fd0 tx=0x7fb9c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.655+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9c404a070 con 0x7fb9b80039e0 2024-04-26T20:00:36.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.871+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9d40b6740 con 0x7fb9b80039e0 2024-04-26T20:00:36.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.872+0000 7fb9cb400640 1 -- 172.21.15.88:0/3301863829 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fb9c4079fb0 con 0x7fb9b80039e0 2024-04-26T20:00:36.873 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:36.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 msgr2=0x7fb9b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 0x7fb9b406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0007fd0 tx=0x7fb9c0009040 comp rx=0 tx=0).stop 2024-04-26T20:00:36.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 msgr2=0x7fb9d414c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:36.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009be0 tx=0x7fb9c4004730 comp rx=0 tx=0).stop 2024-04-26T20:00:36.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 shutdown_connections 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9d415cfa0 0x7fb9d415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb9b406c920 0x7fb9b406ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb9d4156890 0x7fb9d4161e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 --2- 172.21.15.88:0/3301863829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9b80039e0 0x7fb9d414c7c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.874+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 >> 172.21.15.88:0/3301863829 conn(0x7fb9d40b38a0 msgr2=0x7fb9d40b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.875+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 shutdown_connections 2024-04-26T20:00:36.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:36.875+0000 7fb9d8a00640 1 -- 172.21.15.88:0/3301863829 wait complete. 2024-04-26T20:00:36.896 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:36.896 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:36.952 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:37.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.160+0000 7f3f8ea00640 1 Processor -- start 2024-04-26T20:00:37.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.160+0000 7f3f8ea00640 1 -- start start 2024-04-26T20:00:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3f880d0e70 0x7f3f880d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8ea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3f880b5d40 0x7f3f880aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8ea00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f880d1790 con 0x7f3f880b5660 2024-04-26T20:00:37.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8ea00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3f880727f0 con 0x7f3f880b5d40 2024-04-26T20:00:37.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8ea00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3f880b4a10 con 0x7f3f880d0e70 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8cc00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3f880b5660 0x7f3f880ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57006/0 (socket says 172.21.15.88:57006) 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8cc00640 1 -- 172.21.15.88:0/3364005311 learned_addr learned my addr 172.21.15.88:0/3364005311 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.162+0000 7f3f8e000640 1 --2- 172.21.15.88:0/3364005311 >> v2:172.21.15.88:3300/0 conn(0x7f3f880b5d40 0x7f3f880aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f8d600640 1 --2- 172.21.15.88:0/3364005311 >> v2:172.21.15.132:3300/0 conn(0x7f3f880d0e70 0x7f3f880d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f87e00640 1 -- 172.21.15.88:0/3364005311 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 650151581 0 0) 0x7f3f880d1790 con 0x7f3f880b5660 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f87e00640 1 -- 172.21.15.88:0/3364005311 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f70003520 con 0x7f3f880b5660 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f8e000640 1 -- 172.21.15.88:0/3364005311 >> v2:172.21.15.132:3300/0 conn(0x7f3f880d0e70 msgr2=0x7f3f880d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f8e000640 1 --2- 172.21.15.88:0/3364005311 >> v2:172.21.15.132:3300/0 conn(0x7f3f880d0e70 0x7f3f880d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f8e000640 1 -- 172.21.15.88:0/3364005311 >> v1:172.21.15.88:6789/0 conn(0x7f3f880b5660 legacy=0x7f3f880ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.163+0000 7f3f8e000640 1 -- 172.21.15.88:0/3364005311 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f880b4cf0 con 0x7f3f880b5d40 2024-04-26T20:00:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.164+0000 7f3f8e000640 1 --2- 172.21.15.88:0/3364005311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f880aefd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3f880aff60 tx=0x7f3f8002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d933d75dd034ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.164+0000 7f3f87e00640 1 -- 172.21.15.88:0/3364005311 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007740 con 0x7f3f880b5d40 2024-04-26T20:00:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.164+0000 7f3f87e00640 1 -- 172.21.15.88:0/3364005311 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f800078e0 con 0x7f3f880b5d40 2024-04-26T20:00:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.164+0000 7f3f87e00640 1 -- 172.21.15.88:0/3364005311 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007cb0 con 0x7f3f880b5d40 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 -- 172.21.15.88:0/3364005311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 msgr2=0x7f3f880aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/3364005311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f880aefd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3f880aff60 tx=0x7f3f8002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 -- 172.21.15.88:0/3364005311 shutdown_connections 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/3364005311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f880aefd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/3364005311 >> v2:172.21.15.132:3300/0 conn(0x7f3f880d0e70 0x7f3f880d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 -- 172.21.15.88:0/3364005311 >> 172.21.15.88:0/3364005311 conn(0x7f3f8806c740 msgr2=0x7f3f8806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 -- 172.21.15.88:0/3364005311 shutdown_connections 2024-04-26T20:00:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.165+0000 7f3f8ea00640 1 -- 172.21.15.88:0/3364005311 wait complete. 2024-04-26T20:00:37.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.167+0000 7f3f8ea00640 1 Processor -- start 2024-04-26T20:00:37.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.168+0000 7f3f8ea00640 1 -- start start 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f880b5660 0x7f3f8813ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f880d0e70 0x7f3f88151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880b05b0 con 0x7f3f880b5d40 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880b0430 con 0x7f3f880d0e70 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8ea00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8815f1a0 con 0x7f3f880b5660 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40558/0 (socket says 172.21.15.88:40558) 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 -- 172.21.15.88:0/2947952413 learned_addr learned my addr 172.21.15.88:0/2947952413 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8d600640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f880b5660 0x7f3f8813ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 -- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f880b5660 msgr2=0x7f3f8813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f880b5660 0x7f3f8813ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 -- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f880d0e70 msgr2=0x7f3f88151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f880d0e70 0x7f3f88151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.169+0000 7f3f8cc00640 1 -- 172.21.15.88:0/2947952413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f88157c10 con 0x7f3f880b5d40 2024-04-26T20:00:37.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f8cc00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c430 tx=0x7f3f7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78017070 con 0x7f3f880b5d40 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78005ce0 con 0x7f3f880b5d40 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f88151970 con 0x7f3f880b5d40 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780072f0 con 0x7f3f880b5d40 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.170+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f88151f30 con 0x7f3f880b5d40 2024-04-26T20:00:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.171+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f880693c0 con 0x7f3f880b5d40 2024-04-26T20:00:37.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.172+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f7801a070 con 0x7f3f880b5d40 2024-04-26T20:00:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.173+0000 7f3f85600640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.173+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f3f7808b140 con 0x7f3f880b5d40 2024-04-26T20:00:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.174+0000 7f3f8d600640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.174+0000 7f3f8d600640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.178+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f78059440 con 0x7f3f880b5d40 2024-04-26T20:00:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.379+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f880afd40 con 0x7f3f880b5d40 2024-04-26T20:00:37.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.382+0000 7f3f85600640 1 -- 172.21.15.88:0/2947952413 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f3f78058d40 con 0x7f3f880b5d40 2024-04-26T20:00:37.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:37.384 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 msgr2=0x7f3f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c008040 comp rx=0 tx=0).stop 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 msgr2=0x7f3f88150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c430 tx=0x7f3f7800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 shutdown_connections 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f880d0e70 0x7f3f88151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f880b5d40 0x7f3f88150e40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 --2- 172.21.15.88:0/2947952413 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f880b5660 0x7f3f8813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 >> 172.21.15.88:0/2947952413 conn(0x7f3f8806c740 msgr2=0x7f3f8806cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:37.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 shutdown_connections 2024-04-26T20:00:37.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.384+0000 7f3f8ea00640 1 -- 172.21.15.88:0/2947952413 wait complete. 2024-04-26T20:00:37.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:37.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:37.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.609+0000 7f4d89a00640 1 Processor -- start 2024-04-26T20:00:37.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.609+0000 7f4d89a00640 1 -- start start 2024-04-26T20:00:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d89a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4d84160d60 0x7f4d840b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d89a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d84161140 0x7f4d8406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d89a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d840b4d20 con 0x7f4d84146280 2024-04-26T20:00:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d89a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4d840b5000 con 0x7f4d84161140 2024-04-26T20:00:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d89a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4d840b52e0 con 0x7f4d84160d60 2024-04-26T20:00:37.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d83400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d84161140 0x7f4d8406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d83400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4d84161140 0x7f4d8406dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40564/0 (socket says 172.21.15.88:40564) 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d83400640 1 -- 172.21.15.88:0/315283342 learned_addr learned my addr 172.21.15.88:0/315283342 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.611+0000 7f4d83e00640 1 --2- 172.21.15.88:0/315283342 >> v2:172.21.15.132:3300/0 conn(0x7f4d84160d60 0x7f4d840b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.612+0000 7f4d83400640 1 -- 172.21.15.88:0/315283342 >> v2:172.21.15.132:3300/0 conn(0x7f4d84160d60 msgr2=0x7f4d840b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.612+0000 7f4d83400640 1 --2- 172.21.15.88:0/315283342 >> v2:172.21.15.132:3300/0 conn(0x7f4d84160d60 0x7f4d840b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.612+0000 7f4d83400640 1 -- 172.21.15.88:0/315283342 >> v1:172.21.15.132:6789/0 conn(0x7f4d84146280 legacy=0x7f4d840cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.612+0000 7f4d83400640 1 -- 172.21.15.88:0/315283342 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d840cf410 con 0x7f4d84161140 2024-04-26T20:00:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d83400640 1 --2- 172.21.15.88:0/315283342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84161140 0x7f4d8406dee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009870 tx=0x7f4d7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a733e627e66891 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d82a00640 1 -- 172.21.15.88:0/315283342 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70037070 con 0x7f4d84161140 2024-04-26T20:00:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d82a00640 1 -- 172.21.15.88:0/315283342 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d70032440 con 0x7f4d84161140 2024-04-26T20:00:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d82a00640 1 -- 172.21.15.88:0/315283342 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70032810 con 0x7f4d84161140 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d89a00640 1 -- 172.21.15.88:0/315283342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84161140 msgr2=0x7f4d8406dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.613+0000 7f4d89a00640 1 --2- 172.21.15.88:0/315283342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84161140 0x7f4d8406dee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009870 tx=0x7f4d7002a270 comp rx=0 tx=0).stop 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 -- 172.21.15.88:0/315283342 shutdown_connections 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 --2- 172.21.15.88:0/315283342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84161140 0x7f4d8406dee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 --2- 172.21.15.88:0/315283342 >> v2:172.21.15.132:3300/0 conn(0x7f4d84160d60 0x7f4d840b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 -- 172.21.15.88:0/315283342 >> 172.21.15.88:0/315283342 conn(0x7f4d840b2900 msgr2=0x7f4d840b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 -- 172.21.15.88:0/315283342 shutdown_connections 2024-04-26T20:00:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.614+0000 7f4d89a00640 1 -- 172.21.15.88:0/315283342 wait complete. 2024-04-26T20:00:37.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.615+0000 7f4d89a00640 1 Processor -- start 2024-04-26T20:00:37.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 -- start start 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 0x7f4d841597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 0x7f4d840cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8406e420 con 0x7f4d84146280 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4d840b6070 con 0x7f4d84160d60 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.616+0000 7f4d89a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8406e5a0 con 0x7f4d84161140 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40566/0 (socket says 172.21.15.88:40566) 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 -- 172.21.15.88:0/3561781233 learned_addr learned my addr 172.21.15.88:0/3561781233 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d89000640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 0x7f4d840cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83400640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 0x7f4d841597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 -- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 msgr2=0x7f4d840cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 0x7f4d840cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 -- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 msgr2=0x7f4d841597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 0x7f4d841597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83e00640 1 -- 172.21.15.88:0/3561781233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d84143790 con 0x7f4d84146280 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d89000640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 0x7f4d840cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.617+0000 7f4d83400640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 0x7f4d841597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d83e00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d800 tx=0x7f4d7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400b6c0 con 0x7f4d84146280 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d74019070 con 0x7f4d84146280 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400ba80 con 0x7f4d84146280 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d840aa8e0 con 0x7f4d84146280 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.618+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d84135b00 con 0x7f4d84146280 2024-04-26T20:00:37.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.619+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d840693c0 con 0x7f4d84146280 2024-04-26T20:00:37.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.620+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d7400bc20 con 0x7f4d84146280 2024-04-26T20:00:37.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.621+0000 7f4d7be00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 0x7f4d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:37.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.621+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f4d7408c160 con 0x7f4d84146280 2024-04-26T20:00:37.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.621+0000 7f4d83400640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 0x7f4d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:37.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.622+0000 7f4d83400640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 0x7f4d6406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d70034000 tx=0x7f4d7002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.626+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d7401c360 con 0x7f4d84146280 2024-04-26T20:00:37.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.825+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d841355f0 con 0x7f4d84146280 2024-04-26T20:00:37.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.826+0000 7f4d7be00640 1 -- 172.21.15.88:0/3561781233 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4d7405a470 con 0x7f4d84146280 2024-04-26T20:00:37.828 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 msgr2=0x7f4d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 0x7f4d6406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d70034000 tx=0x7f4d7002e000 comp rx=0 tx=0).stop 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 msgr2=0x7f4d84137d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d800 tx=0x7f4d7400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 shutdown_connections 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d6406c8f0 0x7f4d6406edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4d84161140 0x7f4d840cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d84160d60 0x7f4d841597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 --2- 172.21.15.88:0/3561781233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d84146280 0x7f4d84137d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:37.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 >> 172.21.15.88:0/3561781233 conn(0x7f4d840b2900 msgr2=0x7f4d84072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:37.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 shutdown_connections 2024-04-26T20:00:37.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:37.829+0000 7f4d89a00640 1 -- 172.21.15.88:0/3561781233 wait complete. 2024-04-26T20:00:37.852 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:38.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.049+0000 7f3db6400640 1 Processor -- start 2024-04-26T20:00:38.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.050+0000 7f3db6400640 1 -- start start 2024-04-26T20:00:38.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.051+0000 7f3db6400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db00b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.051+0000 7f3db6400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db006ce80 0x7f3db00aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.051+0000 7f3db6400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3db00ade20 con 0x7f3db0073da0 2024-04-26T20:00:38.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.051+0000 7f3db6400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3db00ae100 con 0x7f3db006ce80 2024-04-26T20:00:38.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.051+0000 7f3db6400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00ae3e0 con 0x7f3db0147f20 2024-04-26T20:00:38.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.052+0000 7f3dafe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db006ce80 0x7f3db00aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.052+0000 7f3daf400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db00b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.052+0000 7f3dafe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3db006ce80 0x7f3db00aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:40760/0 (socket says 172.21.15.88:40760) 2024-04-26T20:00:38.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.052+0000 7f3dafe00640 1 -- 172.21.15.88:0/2713451069 learned_addr learned my addr 172.21.15.88:0/2713451069 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3dafe00640 1 -- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 msgr2=0x7f3db00b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3dafe00640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db00b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3dafe00640 1 -- 172.21.15.88:0/2713451069 >> v1:172.21.15.132:6789/0 conn(0x7f3db0073da0 legacy=0x7f3db006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3dafe00640 1 -- 172.21.15.88:0/2713451069 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db00ae6c0 con 0x7f3db006ce80 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3daf400640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db00b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:00:38.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.053+0000 7f3dae000640 1 -- 172.21.15.88:0/2713451069 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436475674 0 0) 0x7f3db00ade20 con 0x7f3db0073da0 2024-04-26T20:00:38.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.055+0000 7f3dafe00640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db00aeec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009870 tx=0x7f3da802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e03463cba06734 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.055+0000 7f3dae000640 1 -- 172.21.15.88:0/2713451069 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8037070 con 0x7f3db006ce80 2024-04-26T20:00:38.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3dae000640 1 -- 172.21.15.88:0/2713451069 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da8032440 con 0x7f3db006ce80 2024-04-26T20:00:38.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3dae000640 1 -- 172.21.15.88:0/2713451069 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8032810 con 0x7f3db006ce80 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 -- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 msgr2=0x7f3db00aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db00aeec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009870 tx=0x7f3da802a250 comp rx=0 tx=0).stop 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 -- 172.21.15.88:0/2713451069 shutdown_connections 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db00aeec0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 --2- 172.21.15.88:0/2713451069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db00b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 -- 172.21.15.88:0/2713451069 >> 172.21.15.88:0/2713451069 conn(0x7f3db00b2d50 msgr2=0x7f3db00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.056+0000 7f3db6400640 1 -- 172.21.15.88:0/2713451069 shutdown_connections 2024-04-26T20:00:38.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.057+0000 7f3db6400640 1 -- 172.21.15.88:0/2713451069 wait complete. 2024-04-26T20:00:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.059+0000 7f3db6400640 1 Processor -- start 2024-04-26T20:00:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.059+0000 7f3db6400640 1 -- start start 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db01426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db0144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3db006e220 con 0x7f3db0073da0 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3db0074200 con 0x7f3db006ce80 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.060+0000 7f3db6400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3db00af400 con 0x7f3db0147f20 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40582/0 (socket says 172.21.15.88:40582) 2024-04-26T20:00:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 -- 172.21.15.88:0/1971653069 learned_addr learned my addr 172.21.15.88:0/1971653069 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daf400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db01426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3dafe00640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db0144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 -- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 msgr2=0x7f3db0144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db0144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 -- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 msgr2=0x7f3db01426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db01426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3daea00640 1 -- 172.21.15.88:0/1971653069 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db0142c30 con 0x7f3db0073da0 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.061+0000 7f3dafe00640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3daea00640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c7e0 tx=0x7f3d9800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003da0 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d98005070 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db01607a0 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98005460 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.062+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db0160fe0 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.063+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db0069650 con 0x7f3db0073da0 2024-04-26T20:00:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.064+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d98005600 con 0x7f3db0073da0 2024-04-26T20:00:38.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.066+0000 7f3da7400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 0x7f3d8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.066+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f3d9808bbd0 con 0x7f3db0073da0 2024-04-26T20:00:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.066+0000 7f3daf400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 0x7f3d8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.067+0000 7f3daf400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 0x7f3d8c06ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009a30 tx=0x7f3da0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.070+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3db01607a0 con 0x7f3db0073da0 2024-04-26T20:00:38.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.269+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3db00b4080 con 0x7f3db0073da0 2024-04-26T20:00:38.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.269+0000 7f3da7400640 1 -- 172.21.15.88:0/1971653069 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3db0142c30 con 0x7f3db0073da0 2024-04-26T20:00:38.271 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:38.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 msgr2=0x7f3d8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 0x7f3d8c06ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009a30 tx=0x7f3da0008040 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 msgr2=0x7f3db015fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800c7e0 tx=0x7f3d9800cca0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 shutdown_connections 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d8c06c920 0x7f3d8c06ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3db0147f20 0x7f3db0144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3db0073da0 0x7f3db015fd60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 --2- 172.21.15.88:0/1971653069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3db006ce80 0x7f3db01426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.272+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 >> 172.21.15.88:0/1971653069 conn(0x7f3db00b2d50 msgr2=0x7f3db00b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.273+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 shutdown_connections 2024-04-26T20:00:38.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.273+0000 7f3db6400640 1 -- 172.21.15.88:0/1971653069 wait complete. 2024-04-26T20:00:38.292 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:38.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.489+0000 7f7d9de00640 1 Processor -- start 2024-04-26T20:00:38.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.489+0000 7f7d9de00640 1 -- start start 2024-04-26T20:00:38.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.490+0000 7f7d9de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 0x7f7d9815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.490+0000 7f7d9de00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d98156750 con 0x7f7d98156370 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.490+0000 7f7d9de00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d9815cd60 con 0x7f7d980d0d60 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.490+0000 7f7d9de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9815d430 con 0x7f7d980d1210 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.491+0000 7f7d9ca00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7d98156370 0x7f7d9815c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:57018/0 (socket says 172.21.15.88:57018) 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.491+0000 7f7d9ca00640 1 -- 172.21.15.88:0/3504689281 learned_addr learned my addr 172.21.15.88:0/3504689281 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:38.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.491+0000 7f7d9d400640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 0x7f7d9815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.491+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3803281535 0 0) 0x7f7d8c001110 con 0x7f7d98156370 2024-04-26T20:00:38.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.491+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d80005730 con 0x7f7d98156370 2024-04-26T20:00:38.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3119184931 0 0) 0x7f7d80005730 con 0x7f7d98156370 2024-04-26T20:00:38.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 msgr2=0x7f7d9815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 0x7f7d9815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 >> v1:172.21.15.132:6789/0 conn(0x7f7d980d0d60 legacy=0x7f7d9815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9815d040 con 0x7f7d98156370 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f7d8c001840 con 0x7f7d98156370 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 >> v1:172.21.15.88:6789/0 conn(0x7f7d98156370 legacy=0x7f7d9815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80005bf0 0x7f7d80006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d80006a10 0x7f7d80006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 0x7f7d800073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9815d430 con 0x7f7d80006a10 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8000dcb0 con 0x7f7d80005bf0 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8000deb0 con 0x7f7d980d1210 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d80006a10 0x7f7d80006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.492+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80005bf0 0x7f7d80006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d97e00640 1 -- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 msgr2=0x7f7d800073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d980d1210 0x7f7d800073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d97e00640 1 -- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80005bf0 msgr2=0x7f7d80006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80005bf0 0x7f7d80006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d97e00640 1 -- 172.21.15.88:0/3504689281 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d80005730 con 0x7f7d80006a10 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.493+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80005bf0 0x7f7d80006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d80006a10 0x7f7d80006e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d84001720 tx=0x7f7d80007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4037f16130e6852 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84001370 con 0x7f7d80006a10 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d840052d0 con 0x7f7d80006a10 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d97400640 1 -- 172.21.15.88:0/3504689281 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8402ab90 con 0x7f7d80006a10 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d9de00640 1 -- 172.21.15.88:0/3504689281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d80006a10 msgr2=0x7f7d80006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3504689281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d80006a10 0x7f7d80006e90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7d84001720 tx=0x7f7d80007a40 comp rx=0 tx=0).stop 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.494+0000 7f7d9d400640 1 -- 172.21.15.88:0/3504689281 reap_dead start 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.495+0000 7f7d9de00640 1 -- 172.21.15.88:0/3504689281 shutdown_connections 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.495+0000 7f7d9de00640 1 -- 172.21.15.88:0/3504689281 >> 172.21.15.88:0/3504689281 conn(0x7f7d9806c530 msgr2=0x7f7d9806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.495+0000 7f7d9de00640 1 -- 172.21.15.88:0/3504689281 shutdown_connections 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.495+0000 7f7d9de00640 1 -- 172.21.15.88:0/3504689281 wait complete. 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.496+0000 7f7d9de00640 1 Processor -- start 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.497+0000 7f7d9de00640 1 -- start start 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.497+0000 7f7d9de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 0x7f7d98151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.497+0000 7f7d9de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 0x7f7d98072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d980d1630 con 0x7f7d980733d0 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9815d960 con 0x7f7d80006a10 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7d980b3cb0 con 0x7f7d981544d0 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40600/0 (socket says 172.21.15.88:40600) 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 -- 172.21.15.88:0/3881428787 learned_addr learned my addr 172.21.15.88:0/3881428787 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 0x7f7d98151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 0x7f7d98072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 -- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 msgr2=0x7f7d98072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 0x7f7d98072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 -- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 msgr2=0x7f7d98151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 0x7f7d98151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9d400640 1 -- 172.21.15.88:0/3881428787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981537c0 con 0x7f7d980733d0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 0x7f7d98151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.498+0000 7f7d97e00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 0x7f7d98072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d9d400640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000edc0 tx=0x7f7d9000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90018020 con 0x7f7d980733d0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9000bb10 con 0x7f7d980733d0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90014400 con 0x7f7d980733d0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d98153aa0 con 0x7f7d980733d0 2024-04-26T20:00:38.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.499+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d98154060 con 0x7f7d980733d0 2024-04-26T20:00:38.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.500+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d980693c0 con 0x7f7d980733d0 2024-04-26T20:00:38.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.501+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d90005d10 con 0x7f7d980733d0 2024-04-26T20:00:38.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.502+0000 7f7d94c00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 0x7f7d7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.502+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f7d9008c1f0 con 0x7f7d980733d0 2024-04-26T20:00:38.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.502+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 0x7f7d7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.503+0000 7f7d9ca00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 0x7f7d7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c001eb0 tx=0x7f7d8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.507+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d90022030 con 0x7f7d980733d0 2024-04-26T20:00:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.704+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d98154260 con 0x7f7d980733d0 2024-04-26T20:00:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.705+0000 7f7d94c00640 1 -- 172.21.15.88:0/3881428787 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f7d9005a580 con 0x7f7d980733d0 2024-04-26T20:00:38.707 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:38.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.707+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 msgr2=0x7f7d7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.707+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 0x7f7d7806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c001eb0 tx=0x7f7d8c009040 comp rx=0 tx=0).stop 2024-04-26T20:00:38.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 msgr2=0x7f7d981e5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000edc0 tx=0x7f7d9000b480 comp rx=0 tx=0).stop 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 shutdown_connections 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d980733d0 0x7f7d981e5aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7d7806c8f0 0x7f7d7806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7d981544d0 0x7f7d98072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 --2- 172.21.15.88:0/3881428787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d80006a10 0x7f7d98151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 >> 172.21.15.88:0/3881428787 conn(0x7f7d9806c530 msgr2=0x7f7d9815fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 shutdown_connections 2024-04-26T20:00:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.708+0000 7f7d9de00640 1 -- 172.21.15.88:0/3881428787 wait complete. 2024-04-26T20:00:38.729 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:38.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.927+0000 7f6922800640 1 Processor -- start 2024-04-26T20:00:38.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.927+0000 7f6922800640 1 -- start start 2024-04-26T20:00:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.928+0000 7f6922800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f691c06ce80 0x7f691c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f6922800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 0x7f691c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f6922800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691c0af9a0 con 0x7f691c15cdf0 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f6922800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f691c0ae2c0 con 0x7f691c06ce80 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f6922800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f691c0b05b0 con 0x7f691c0cd650 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691aa00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f691c06ce80 0x7f691c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f691c15cdf0 0x7f691c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47930/0 (socket says 172.21.15.88:47930) 2024-04-26T20:00:38.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691b400640 1 -- 172.21.15.88:0/3953952696 learned_addr learned my addr 172.21.15.88:0/3953952696 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691be00640 1 --2- 172.21.15.88:0/3953952696 >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 0x7f691c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691aa00640 1 -- 172.21.15.88:0/3953952696 >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 msgr2=0x7f691c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691aa00640 1 --2- 172.21.15.88:0/3953952696 >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 0x7f691c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691aa00640 1 -- 172.21.15.88:0/3953952696 >> v1:172.21.15.88:6789/0 conn(0x7f691c15cdf0 legacy=0x7f691c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691a000640 1 -- 172.21.15.88:0/3953952696 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538325435 0 0) 0x7f691c0af9a0 con 0x7f691c15cdf0 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.929+0000 7f691aa00640 1 -- 172.21.15.88:0/3953952696 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c0afc80 con 0x7f691c06ce80 2024-04-26T20:00:38.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.930+0000 7f691be00640 1 --2- 172.21.15.88:0/3953952696 >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 0x7f691c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f691aa00640 1 --2- 172.21.15.88:0/3953952696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c0add80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f691c0b0520 tx=0x7f690802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c369b5339450fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f691a000640 1 -- 172.21.15.88:0/3953952696 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908037070 con 0x7f691c06ce80 2024-04-26T20:00:38.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f691a000640 1 -- 172.21.15.88:0/3953952696 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6908032440 con 0x7f691c06ce80 2024-04-26T20:00:38.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f691a000640 1 -- 172.21.15.88:0/3953952696 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908032810 con 0x7f691c06ce80 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f6922800640 1 -- 172.21.15.88:0/3953952696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 msgr2=0x7f691c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.931+0000 7f6922800640 1 --2- 172.21.15.88:0/3953952696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c0add80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f691c0b0520 tx=0x7f690802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 -- 172.21.15.88:0/3953952696 shutdown_connections 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 --2- 172.21.15.88:0/3953952696 >> v2:172.21.15.132:3300/0 conn(0x7f691c0cd650 0x7f691c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 --2- 172.21.15.88:0/3953952696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c0add80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 -- 172.21.15.88:0/3953952696 >> 172.21.15.88:0/3953952696 conn(0x7f691c0b2d50 msgr2=0x7f691c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 -- 172.21.15.88:0/3953952696 shutdown_connections 2024-04-26T20:00:38.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.932+0000 7f6922800640 1 -- 172.21.15.88:0/3953952696 wait complete. 2024-04-26T20:00:38.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.934+0000 7f6922800640 1 Processor -- start 2024-04-26T20:00:38.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.934+0000 7f6922800640 1 -- start start 2024-04-26T20:00:38.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f691c0cd650 0x7f691c156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 0x7f691c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f691c15d1d0 con 0x7f691c06ce80 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f691c0b0240 con 0x7f691c15cdf0 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f6922800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f691c15d350 con 0x7f691c0cd650 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47212/0 (socket says 172.21.15.88:47212) 2024-04-26T20:00:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 -- 172.21.15.88:0/1632426220 learned_addr learned my addr 172.21.15.88:0/1632426220 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:38.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691aa00640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f691c0cd650 0x7f691c156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 -- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f691c0cd650 msgr2=0x7f691c156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691be00640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 0x7f691c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f691c0cd650 0x7f691c156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 -- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 msgr2=0x7f691c15b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 0x7f691c15b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691b400640 1 -- 172.21.15.88:0/1632426220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c06e550 con 0x7f691c06ce80 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.935+0000 7f691be00640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 0x7f691c15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f691b400640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c460 tx=0x7f690c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c018070 con 0x7f691c06ce80 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c007500 con 0x7f691c06ce80 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c004a20 con 0x7f691c06ce80 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c06e7d0 con 0x7f691c06ce80 2024-04-26T20:00:38.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.936+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c06ed90 con 0x7f691c06ce80 2024-04-26T20:00:38.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.938+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c069540 con 0x7f691c06ce80 2024-04-26T20:00:38.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.938+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f690c004bc0 con 0x7f691c06ce80 2024-04-26T20:00:38.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.939+0000 7f6913400640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 0x7f68fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:38.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.939+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f690c08afe0 con 0x7f691c06ce80 2024-04-26T20:00:38.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.940+0000 7f691aa00640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 0x7f68fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:38.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.940+0000 7f691aa00640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 0x7f68fc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6908034000 tx=0x7f690802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:38.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:38.944+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f691c06e550 con 0x7f691c06ce80 2024-04-26T20:00:39.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.157+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f691c156880 con 0x7f691c06ce80 2024-04-26T20:00:39.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.158+0000 7f6913400640 1 -- 172.21.15.88:0/1632426220 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f690c059290 con 0x7f691c06ce80 2024-04-26T20:00:39.159 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.160+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 msgr2=0x7f68fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.160+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 0x7f68fc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6908034000 tx=0x7f690802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.160+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 msgr2=0x7f691c06e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.160+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c460 tx=0x7f690c00c920 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 shutdown_connections 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f691c15cdf0 0x7f691c15b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68fc06c8f0 0x7f68fc06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f691c0cd650 0x7f691c156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 --2- 172.21.15.88:0/1632426220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f691c06ce80 0x7f691c06e010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 >> 172.21.15.88:0/1632426220 conn(0x7f691c0b2d50 msgr2=0x7f691c06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:39.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 shutdown_connections 2024-04-26T20:00:39.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.161+0000 7f6922800640 1 -- 172.21.15.88:0/1632426220 wait complete. 2024-04-26T20:00:39.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi132.front.sepia.ceph.com... 2024-04-26T20:00:39.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:39.181 DEBUG:teuthology.orchestra.run.smithi132:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:00:39.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:39.271 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:00:39.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:39.271 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:00:39.337 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:39.337 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:39.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:39.394 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:39.446 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:39.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:00:39.524 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:00:39.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.639+0000 7fdb9dc00640 1 Processor -- start 2024-04-26T20:00:39.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.639+0000 7fdb9dc00640 1 -- start start 2024-04-26T20:00:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.640+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.640+0000 7fdb9dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdb9815cfd0 0x7fdb9806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.640+0000 7fdb9dc00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb980ae850 con 0x7fdb9806c770 2024-04-26T20:00:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.640+0000 7fdb9dc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdb980af460 con 0x7fdb9815cfd0 2024-04-26T20:00:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.640+0000 7fdb9dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980aeb30 con 0x7fdb9815cb70 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb96a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdb9806c770 0x7fdb9806cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47946/0 (socket says 172.21.15.88:47946) 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb96a00640 1 -- 172.21.15.88:0/1276587259 learned_addr learned my addr 172.21.15.88:0/1276587259 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb97400640 1 --2- 172.21.15.88:0/1276587259 >> v2:172.21.15.88:3300/0 conn(0x7fdb9815cfd0 0x7fdb9806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb97400640 1 -- 172.21.15.88:0/1276587259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 msgr2=0x7fdb9806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb97400640 1 --2- 172.21.15.88:0/1276587259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb97400640 1 -- 172.21.15.88:0/1276587259 >> v1:172.21.15.88:6789/0 conn(0x7fdb9806c770 legacy=0x7fdb9806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.641+0000 7fdb97400640 1 -- 172.21.15.88:0/1276587259 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980aee10 con 0x7fdb9815cfd0 2024-04-26T20:00:39.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.642+0000 7fdb97400640 1 --2- 172.21.15.88:0/1276587259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9815cfd0 0x7fdb9806f3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009d90 tx=0x7fdb9002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b16435a3922204 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:39.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb96000640 1 -- 172.21.15.88:0/1276587259 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90032770 con 0x7fdb9815cfd0 2024-04-26T20:00:39.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb96000640 1 -- 172.21.15.88:0/1276587259 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90038070 con 0x7fdb9815cfd0 2024-04-26T20:00:39.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb96000640 1 -- 172.21.15.88:0/1276587259 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90032b30 con 0x7fdb9815cfd0 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 -- 172.21.15.88:0/1276587259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9815cfd0 msgr2=0x7fdb9806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/1276587259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9815cfd0 0x7fdb9806f3a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009d90 tx=0x7fdb9002c870 comp rx=0 tx=0).stop 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 -- 172.21.15.88:0/1276587259 shutdown_connections 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/1276587259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9815cfd0 0x7fdb9806f3a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/1276587259 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9806ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 -- 172.21.15.88:0/1276587259 >> 172.21.15.88:0/1276587259 conn(0x7fdb980b2d50 msgr2=0x7fdb980b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 -- 172.21.15.88:0/1276587259 shutdown_connections 2024-04-26T20:00:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.643+0000 7fdb9dc00640 1 -- 172.21.15.88:0/1276587259 wait complete. 2024-04-26T20:00:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.645+0000 7fdb9dc00640 1 Processor -- start 2024-04-26T20:00:39.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.645+0000 7fdb9dc00640 1 -- start start 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9815cfd0 0x7fdb9813e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9815c780 con 0x7fdb9806c770 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9815c600 con 0x7fdb9815cfd0 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb9dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdb980af8f0 con 0x7fdb9815cb70 2024-04-26T20:00:39.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:39.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb96a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47220/0 (socket says 172.21.15.88:47220) 2024-04-26T20:00:39.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.646+0000 7fdb96a00640 1 -- 172.21.15.88:0/3464186189 learned_addr learned my addr 172.21.15.88:0/3464186189 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb97400640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9815cfd0 0x7fdb9813e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb8fe00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 -- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 msgr2=0x7fdb9813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 -- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9815cfd0 msgr2=0x7fdb9813e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9815cfd0 0x7fdb9813e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 -- 172.21.15.88:0/3464186189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb980720b0 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb8fe00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.647+0000 7fdb96a00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c430 tx=0x7fdb8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.648+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88017070 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.648+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb88005ce0 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.648+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880072f0 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.648+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb98159ca0 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.648+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9813e280 con 0x7fdb9806c770 2024-04-26T20:00:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.649+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb98069540 con 0x7fdb9806c770 2024-04-26T20:00:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.650+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb8801a070 con 0x7fdb9806c770 2024-04-26T20:00:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.651+0000 7fdb8ea00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 0x7fdb7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.651+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fdb8808b0c0 con 0x7fdb9806c770 2024-04-26T20:00:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.651+0000 7fdb8fe00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 0x7fdb7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.652+0000 7fdb8fe00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 0x7fdb7406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006fd0 tx=0x7fdb80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:39.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.655+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb880593c0 con 0x7fdb9806c770 2024-04-26T20:00:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.859+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb9806ec40 con 0x7fdb9806c770 2024-04-26T20:00:39.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.863+0000 7fdb8ea00640 1 -- 172.21.15.88:0/3464186189 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7fdb88082e20 con 0x7fdb9806c770 2024-04-26T20:00:39.865 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:39.865 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 msgr2=0x7fdb7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 0x7fdb7406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006fd0 tx=0x7fdb80008040 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 msgr2=0x7fdb98140090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c430 tx=0x7fdb8800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 shutdown_connections 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9815cfd0 0x7fdb9813e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb7406c8f0 0x7fdb7406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb9815cb70 0x7fdb9813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 --2- 172.21.15.88:0/3464186189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9806c770 0x7fdb98140090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 >> 172.21.15.88:0/3464186189 conn(0x7fdb980b2d50 msgr2=0x7fdb981537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 shutdown_connections 2024-04-26T20:00:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:39.866+0000 7fdb9dc00640 1 -- 172.21.15.88:0/3464186189 wait complete. 2024-04-26T20:00:39.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:39.887 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:40.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.138+0000 7f4dc3200640 1 Processor -- start 2024-04-26T20:00:40.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.138+0000 7f4dc3200640 1 -- start start 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4dbc15d100 0x7f4dbc06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc0af800 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc0ae340 con 0x7f4dbc15d100 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dc3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc0afaa0 con 0x7f4dbc0add80 2024-04-26T20:00:40.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dbb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dbb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47234/0 (socket says 172.21.15.88:47234) 2024-04-26T20:00:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.140+0000 7f4dbb400640 1 -- 172.21.15.88:0/2338644253 learned_addr learned my addr 172.21.15.88:0/2338644253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbbe00640 1 --2- 172.21.15.88:0/2338644253 >> v2:172.21.15.132:3300/0 conn(0x7f4dbc15d100 0x7f4dbc06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dc1200640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbb400640 1 -- 172.21.15.88:0/2338644253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 msgr2=0x7f4dbc072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbb400640 1 -- 172.21.15.88:0/2338644253 >> v2:172.21.15.132:3300/0 conn(0x7f4dbc15d100 msgr2=0x7f4dbc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2338644253 >> v2:172.21.15.132:3300/0 conn(0x7f4dbc15d100 0x7f4dbc06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.141+0000 7f4dbb400640 1 -- 172.21.15.88:0/2338644253 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc0afd80 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.142+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0730d0 tx=0x7f4da802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a94de3cdcefb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.142+0000 7f4dbaa00640 1 -- 172.21.15.88:0/2338644253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8037070 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.142+0000 7f4dbaa00640 1 -- 172.21.15.88:0/2338644253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8032440 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.142+0000 7f4dbaa00640 1 -- 172.21.15.88:0/2338644253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8032810 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 -- 172.21.15.88:0/2338644253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 msgr2=0x7f4dbc06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0730d0 tx=0x7f4da802a140 comp rx=0 tx=0).stop 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 -- 172.21.15.88:0/2338644253 shutdown_connections 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2338644253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc06ccb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2338644253 >> v2:172.21.15.132:3300/0 conn(0x7f4dbc15d100 0x7f4dbc06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 -- 172.21.15.88:0/2338644253 >> 172.21.15.88:0/2338644253 conn(0x7f4dbc0b2d50 msgr2=0x7f4dbc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 -- 172.21.15.88:0/2338644253 shutdown_connections 2024-04-26T20:00:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.143+0000 7f4dc3200640 1 -- 172.21.15.88:0/2338644253 wait complete. 2024-04-26T20:00:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.145+0000 7f4dc3200640 1 Processor -- start 2024-04-26T20:00:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.145+0000 7f4dc3200640 1 -- start start 2024-04-26T20:00:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4dbc15d100 0x7f4dbc157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc09ee50 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc0b0340 con 0x7f4dbc15d100 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dc3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc160b70 con 0x7f4dbc0add80 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dbb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dbb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47242/0 (socket says 172.21.15.88:47242) 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.146+0000 7f4dbb400640 1 -- 172.21.15.88:0/2572562247 learned_addr learned my addr 172.21.15.88:0/2572562247 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbbe00640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dc1200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4dbc15d100 0x7f4dbc157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 -- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 msgr2=0x7f4dbc14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc14e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 -- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4dbc15d100 msgr2=0x7f4dbc157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4dbc15d100 0x7f4dbc157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 -- 172.21.15.88:0/2572562247 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc14f050 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4dbb400640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4da802a180 tx=0x7f4da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.147+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8041020 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.148+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da80306c0 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.148+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc09d0e0 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.148+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8037040 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.148+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc160120 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.149+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc069650 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.150+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4da8044020 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.150+0000 7f4db3e00640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 0x7f4d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.151+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f4da80ac440 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.151+0000 7f4dbbe00640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 0x7f4d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.151+0000 7f4dbbe00640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 0x7f4d9c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.155+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4da804b030 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.353+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4dbc0d0630 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.354+0000 7f4db3e00640 1 -- 172.21.15.88:0/2572562247 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4dbc14f050 con 0x7f4dbc0b5ca0 2024-04-26T20:00:40.356 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:40.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.356+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 msgr2=0x7f4d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.356+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 0x7f4d9c06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac006fd0 tx=0x7f4dac008040 comp rx=0 tx=0).stop 2024-04-26T20:00:40.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 msgr2=0x7f4dbc157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4da802a180 tx=0x7f4da80046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 shutdown_connections 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4dbc15d100 0x7f4dbc157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4dbc0b5ca0 0x7f4dbc157170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4d9c06c8f0 0x7f4d9c06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 --2- 172.21.15.88:0/2572562247 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4dbc0add80 0x7f4dbc14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 >> 172.21.15.88:0/2572562247 conn(0x7f4dbc0b2d50 msgr2=0x7f4dbc0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 shutdown_connections 2024-04-26T20:00:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.357+0000 7f4dc3200640 1 -- 172.21.15.88:0/2572562247 wait complete. 2024-04-26T20:00:40.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:40.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.565+0000 7f8704c00640 1 Processor -- start 2024-04-26T20:00:40.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.565+0000 7f8704c00640 1 -- start start 2024-04-26T20:00:40.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f8704c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87000b4ed0 0x7f87000b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f8704c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 0x7f87000ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f8704c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870006e320 con 0x7f8700071e80 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f8704c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f870006e600 con 0x7f87000b4ed0 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f8704c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f870006e8e0 con 0x7f8700072970 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.567+0000 7f86f6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87000b4ed0 0x7f87000b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87000b4ed0 0x7f87000b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47250/0 (socket says 172.21.15.88:47250) 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 -- 172.21.15.88:0/2076565526 learned_addr learned my addr 172.21.15.88:0/2076565526 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86ffe00640 1 --2- 172.21.15.88:0/2076565526 >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 0x7f87000ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 -- 172.21.15.88:0/2076565526 >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 msgr2=0x7f87000ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 --2- 172.21.15.88:0/2076565526 >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 0x7f87000ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 -- 172.21.15.88:0/2076565526 >> v1:172.21.15.132:6789/0 conn(0x7f8700071e80 legacy=0x7f8700072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86f6a00640 1 -- 172.21.15.88:0/2076565526 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870006ebc0 con 0x7f87000b4ed0 2024-04-26T20:00:40.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.568+0000 7f86ffe00640 1 --2- 172.21.15.88:0/2076565526 >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 0x7f87000ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:40.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.569+0000 7f86f6a00640 1 --2- 172.21.15.88:0/2076565526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87000b3ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8700073d20 tx=0x7f86e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1699f85b52c1fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.569+0000 7f86fea00640 1 -- 172.21.15.88:0/2076565526 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8037070 con 0x7f87000b4ed0 2024-04-26T20:00:40.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.569+0000 7f86fea00640 1 -- 172.21.15.88:0/2076565526 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e8032440 con 0x7f87000b4ed0 2024-04-26T20:00:40.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.569+0000 7f86fea00640 1 -- 172.21.15.88:0/2076565526 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8032810 con 0x7f87000b4ed0 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 -- 172.21.15.88:0/2076565526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 msgr2=0x7f87000b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 --2- 172.21.15.88:0/2076565526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87000b3ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8700073d20 tx=0x7f86e802a230 comp rx=0 tx=0).stop 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 -- 172.21.15.88:0/2076565526 shutdown_connections 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 --2- 172.21.15.88:0/2076565526 >> v2:172.21.15.132:3300/0 conn(0x7f8700072970 0x7f87000ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 --2- 172.21.15.88:0/2076565526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87000b3ee0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 -- 172.21.15.88:0/2076565526 >> 172.21.15.88:0/2076565526 conn(0x7f87000b2900 msgr2=0x7f87000b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 -- 172.21.15.88:0/2076565526 shutdown_connections 2024-04-26T20:00:40.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.570+0000 7f8704c00640 1 -- 172.21.15.88:0/2076565526 wait complete. 2024-04-26T20:00:40.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.572+0000 7f8704c00640 1 Processor -- start 2024-04-26T20:00:40.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.572+0000 7f8704c00640 1 -- start start 2024-04-26T20:00:40.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 0x7f87001f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 0x7f8700147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f870006f0e0 con 0x7f87000b4ed0 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f87000b5ff0 con 0x7f8700072970 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f8704c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f870006f260 con 0x7f8700071e80 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f86ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47256/0 (socket says 172.21.15.88:47256) 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.573+0000 7f86ffe00640 1 -- 172.21.15.88:0/3910821144 learned_addr learned my addr 172.21.15.88:0/3910821144 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86f6a00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 0x7f8700147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ff400640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 0x7f87001f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 -- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 msgr2=0x7f87001f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 0x7f87001f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 -- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 msgr2=0x7f8700147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 0x7f8700147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 -- 172.21.15.88:0/3910821144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87001f5aa0 con 0x7f87000b4ed0 2024-04-26T20:00:40.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86f6a00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 0x7f8700147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ff400640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 0x7f87001f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.574+0000 7f86ffe00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ce40 tx=0x7f86f800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.575+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f801a070 con 0x7f87000b4ed0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.575+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f800d840 con 0x7f87000b4ed0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.575+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f800dc30 con 0x7f87000b4ed0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.575+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87001598e0 con 0x7f87000b4ed0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.575+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8700159ea0 con 0x7f87000b4ed0 2024-04-26T20:00:40.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.576+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87000693c0 con 0x7f87000b4ed0 2024-04-26T20:00:40.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.577+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86f801d070 con 0x7f87000b4ed0 2024-04-26T20:00:40.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.579+0000 7f86f7e00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 0x7f86dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:40.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.579+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f86f808c3f0 con 0x7f87000b4ed0 2024-04-26T20:00:40.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.579+0000 7f86ff400640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 0x7f86dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:40.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.579+0000 7f86ff400640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 0x7f86dc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f870015b100 tx=0x7f86f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:40.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.583+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f87001f5aa0 con 0x7f87000b4ed0 2024-04-26T20:00:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.784+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8700155b40 con 0x7f87000b4ed0 2024-04-26T20:00:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.785+0000 7f86f7e00640 1 -- 172.21.15.88:0/3910821144 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f86f805a790 con 0x7f87000b4ed0 2024-04-26T20:00:40.787 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 msgr2=0x7f86dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 0x7f86dc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f870015b100 tx=0x7f86f0009040 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 msgr2=0x7f87001618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86f800ce40 tx=0x7f86f800a710 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 shutdown_connections 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87000b4ed0 0x7f87001618f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700072970 0x7f8700147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86dc06c8f0 0x7f86dc06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 --2- 172.21.15.88:0/3910821144 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8700071e80 0x7f87001f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 >> 172.21.15.88:0/3910821144 conn(0x7f87000b2900 msgr2=0x7f87000ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:40.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 shutdown_connections 2024-04-26T20:00:40.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:40.788+0000 7f8704c00640 1 -- 172.21.15.88:0/3910821144 wait complete. 2024-04-26T20:00:40.809 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.010+0000 7fb5bd800640 1 Processor -- start 2024-04-26T20:00:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.010+0000 7fb5bd800640 1 -- start start 2024-04-26T20:00:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 0x7fb5b806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5b80add80 0x7fb5b8072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b80af800 con 0x7fb5b80add80 2024-04-26T20:00:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b80ae340 con 0x7fb5b815d100 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5bd800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b80afaa0 con 0x7fb5b80b5ca0 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b7400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5b80add80 0x7fb5b8072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b7400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5b80add80 0x7fb5b8072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47262/0 (socket says 172.21.15.88:47262) 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b7400640 1 -- 172.21.15.88:0/3946530601 learned_addr learned my addr 172.21.15.88:0/3946530601 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b6a00640 1 --2- 172.21.15.88:0/3946530601 >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 0x7fb5b806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b6000640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.012+0000 7fb5b7400640 1 -- 172.21.15.88:0/3946530601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 msgr2=0x7fb5b806ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b7400640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b806ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b7400640 1 -- 172.21.15.88:0/3946530601 >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 msgr2=0x7fb5b806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b7400640 1 --2- 172.21.15.88:0/3946530601 >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 0x7fb5b806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b7400640 1 -- 172.21.15.88:0/3946530601 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b80afd80 con 0x7fb5b80add80 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b6a00640 1 --2- 172.21.15.88:0/3946530601 >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 0x7fb5b806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:41.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.013+0000 7fb5b6000640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:41.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5b7400640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b80add80 0x7fb5b8072010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80730d0 tx=0x7fb5b002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22eb323c72eace7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5b5600640 1 -- 172.21.15.88:0/3946530601 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0036070 con 0x7fb5b80add80 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5b5600640 1 -- 172.21.15.88:0/3946530601 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b0031600 con 0x7fb5b80add80 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5b5600640 1 -- 172.21.15.88:0/3946530601 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00319d0 con 0x7fb5b80add80 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5bd800640 1 -- 172.21.15.88:0/3946530601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b80add80 msgr2=0x7fb5b8072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.014+0000 7fb5bd800640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b80add80 0x7fb5b8072010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80730d0 tx=0x7fb5b002a610 comp rx=0 tx=0).stop 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 -- 172.21.15.88:0/3946530601 shutdown_connections 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b80add80 0x7fb5b8072010 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 --2- 172.21.15.88:0/3946530601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b806ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 --2- 172.21.15.88:0/3946530601 >> v2:172.21.15.132:3300/0 conn(0x7fb5b815d100 0x7fb5b806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 -- 172.21.15.88:0/3946530601 >> 172.21.15.88:0/3946530601 conn(0x7fb5b80b2d50 msgr2=0x7fb5b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 -- 172.21.15.88:0/3946530601 shutdown_connections 2024-04-26T20:00:41.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.015+0000 7fb5bd800640 1 -- 172.21.15.88:0/3946530601 wait complete. 2024-04-26T20:00:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.016+0000 7fb5bd800640 1 Processor -- start 2024-04-26T20:00:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.017+0000 7fb5bd800640 1 -- start start 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.017+0000 7fb5bd800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5b80add80 0x7fb5b814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5bd800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b8151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5bd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5bd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b809ee50 con 0x7fb5b815d100 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5bd800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b80b0340 con 0x7fb5b80add80 2024-04-26T20:00:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5bd800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b8160b70 con 0x7fb5b80b5ca0 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47276/0 (socket says 172.21.15.88:47276) 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 -- 172.21.15.88:0/2216043932 learned_addr learned my addr 172.21.15.88:0/2216043932 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b6a00640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5b80add80 0x7fb5b814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b6000640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b8151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 -- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 msgr2=0x7fb5b8151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b8151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 -- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5b80add80 msgr2=0x7fb5b814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5b80add80 0x7fb5b814f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.018+0000 7fb5b7400640 1 -- 172.21.15.88:0/2216043932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b8152560 con 0x7fb5b815d100 2024-04-26T20:00:41.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5b7400640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009f40 tx=0x7fb5b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0041020 con 0x7fb5b815d100 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b0007780 con 0x7fb5b815d100 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0036040 con 0x7fb5b815d100 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b8152840 con 0x7fb5b815d100 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.019+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b8152e00 con 0x7fb5b815d100 2024-04-26T20:00:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.020+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b8069650 con 0x7fb5b815d100 2024-04-26T20:00:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.021+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5b0044020 con 0x7fb5b815d100 2024-04-26T20:00:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.023+0000 7fb5aea00640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 0x7fb59806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.023+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fb5b00acaa0 con 0x7fb5b815d100 2024-04-26T20:00:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.023+0000 7fb5b6a00640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 0x7fb59806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.024+0000 7fb5b6a00640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 0x7fb59806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800a8d0 tx=0x7fb5a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.027+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5b004b030 con 0x7fb5b815d100 2024-04-26T20:00:41.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.232+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5b806d1c0 con 0x7fb5b815d100 2024-04-26T20:00:41.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.238+0000 7fb5aea00640 1 -- 172.21.15.88:0/2216043932 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fb5b007ada0 con 0x7fb5b815d100 2024-04-26T20:00:41.240 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 msgr2=0x7fb59806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 0x7fb59806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800a8d0 tx=0x7fb5a8008040 comp rx=0 tx=0).stop 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 msgr2=0x7fb5b809d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009f40 tx=0x7fb5b0004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 shutdown_connections 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5b815d100 0x7fb5b809d040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb59806c8f0 0x7fb59806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5b80b5ca0 0x7fb5b8151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 --2- 172.21.15.88:0/2216043932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5b80add80 0x7fb5b814f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 >> 172.21.15.88:0/2216043932 conn(0x7fb5b80b2d50 msgr2=0x7fb5b80ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 shutdown_connections 2024-04-26T20:00:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.241+0000 7fb5bd800640 1 -- 172.21.15.88:0/2216043932 wait complete. 2024-04-26T20:00:41.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:41.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.444+0000 7f86c2e00640 1 Processor -- start 2024-04-26T20:00:41.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.444+0000 7f86c2e00640 1 -- start start 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f86bc06c770 0x7f86bc06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86bc15cb70 0x7f86bc073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc15cfd0 0x7f86bc073660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc0af460 con 0x7f86bc15cb70 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc06d090 con 0x7f86bc06c770 2024-04-26T20:00:41.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86c2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc0ae850 con 0x7f86bc15cfd0 2024-04-26T20:00:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86bc15cb70 0x7f86bc073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f86bc15cb70 0x7f86bc073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47286/0 (socket says 172.21.15.88:47286) 2024-04-26T20:00:41.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 -- 172.21.15.88:0/3962784760 learned_addr learned my addr 172.21.15.88:0/3962784760 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 -- 172.21.15.88:0/3962784760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc15cfd0 msgr2=0x7f86bc073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 --2- 172.21.15.88:0/3962784760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc15cfd0 0x7f86bc073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 -- 172.21.15.88:0/3962784760 >> v2:172.21.15.132:3300/0 conn(0x7f86bc06c770 msgr2=0x7f86bc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.446+0000 7f86bb400640 1 --2- 172.21.15.88:0/3962784760 >> v2:172.21.15.132:3300/0 conn(0x7f86bc06c770 0x7f86bc06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.447+0000 7f86bb400640 1 -- 172.21.15.88:0/3962784760 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0aeb30 con 0x7f86bc15cb70 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86bb400640 1 --2- 172.21.15.88:0/3962784760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cb70 0x7f86bc073120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0741f0 tx=0x7f86a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212ac02a76b1af75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86baa00640 1 -- 172.21.15.88:0/3962784760 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4037070 con 0x7f86bc15cb70 2024-04-26T20:00:41.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86baa00640 1 -- 172.21.15.88:0/3962784760 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a4032440 con 0x7f86bc15cb70 2024-04-26T20:00:41.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86baa00640 1 -- 172.21.15.88:0/3962784760 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4032810 con 0x7f86bc15cb70 2024-04-26T20:00:41.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86c2e00640 1 -- 172.21.15.88:0/3962784760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cb70 msgr2=0x7f86bc073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.448+0000 7f86c2e00640 1 --2- 172.21.15.88:0/3962784760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cb70 0x7f86bc073120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0741f0 tx=0x7f86a402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 -- 172.21.15.88:0/3962784760 shutdown_connections 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 --2- 172.21.15.88:0/3962784760 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc15cfd0 0x7f86bc073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 --2- 172.21.15.88:0/3962784760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cb70 0x7f86bc073120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 --2- 172.21.15.88:0/3962784760 >> v2:172.21.15.132:3300/0 conn(0x7f86bc06c770 0x7f86bc06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 -- 172.21.15.88:0/3962784760 >> 172.21.15.88:0/3962784760 conn(0x7f86bc0b2d50 msgr2=0x7f86bc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 -- 172.21.15.88:0/3962784760 shutdown_connections 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.449+0000 7f86c2e00640 1 -- 172.21.15.88:0/3962784760 wait complete. 2024-04-26T20:00:41.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.450+0000 7f86c2e00640 1 Processor -- start 2024-04-26T20:00:41.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.450+0000 7f86c2e00640 1 -- start start 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 0x7f86bc1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 0x7f86bc0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc15c600 con 0x7f86bc15cfd0 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc0af0f0 con 0x7f86bc15cb70 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.451+0000 7f86c2e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc15c780 con 0x7f86bc06c770 2024-04-26T20:00:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47298/0 (socket says 172.21.15.88:47298) 2024-04-26T20:00:41.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 -- 172.21.15.88:0/2796246100 learned_addr learned my addr 172.21.15.88:0/2796246100 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86bb400640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 0x7f86bc0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86bbe00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 0x7f86bc1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 -- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 msgr2=0x7f86bc1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 0x7f86bc1f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 -- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 msgr2=0x7f86bc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 0x7f86bc0cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 -- 172.21.15.88:0/2796246100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0d0680 con 0x7f86bc15cfd0 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86bbe00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 0x7f86bc1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:41.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86bb400640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 0x7f86bc0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.452+0000 7f86c0e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86b400b970 tx=0x7f86b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.453+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4019070 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.453+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b400cb60 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.453+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4014440 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.453+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc0d00c0 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.453+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1f35d0 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.454+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc069540 con 0x7f86bc15cfd0 2024-04-26T20:00:41.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.455+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86b401c070 con 0x7f86bc15cfd0 2024-04-26T20:00:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.456+0000 7f86b3e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 0x7f86980731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.456+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f86b408b9c0 con 0x7f86bc15cfd0 2024-04-26T20:00:41.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.456+0000 7f86bbe00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 0x7f86980731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.457+0000 7f86bbe00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 0x7f86980731c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00a870 tx=0x7f86ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.460+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86b4059cc0 con 0x7f86bc15cfd0 2024-04-26T20:00:41.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.656+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86bc0cd430 con 0x7f86bc15cfd0 2024-04-26T20:00:41.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.657+0000 7f86b3e00640 1 -- 172.21.15.88:0/2796246100 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f86b40595c0 con 0x7f86bc15cfd0 2024-04-26T20:00:41.658 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 msgr2=0x7f86980731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 0x7f86980731c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00a870 tx=0x7f86ac008040 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 msgr2=0x7f86bc0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f86b400b970 tx=0x7f86b400be30 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 shutdown_connections 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86bc15cfd0 0x7f86bc0b4f60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86bc15cb70 0x7f86bc0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8698070d00 0x7f86980731c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 --2- 172.21.15.88:0/2796246100 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f86bc06c770 0x7f86bc1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 >> 172.21.15.88:0/2796246100 conn(0x7f86bc0b2d50 msgr2=0x7f86bc160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 shutdown_connections 2024-04-26T20:00:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.659+0000 7f86c2e00640 1 -- 172.21.15.88:0/2796246100 wait complete. 2024-04-26T20:00:41.678 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:00:41.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:41.726 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:41.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.947+0000 7f6365c00640 1 Processor -- start 2024-04-26T20:00:41.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.947+0000 7f6365c00640 1 -- start start 2024-04-26T20:00:41.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6360146680 0x7f63600b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63600b4b30 0x7f63600d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6360146a60 con 0x7f63600d0e70 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f636006e970 con 0x7f63600b4b30 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f63600d17f0 con 0x7f6360146680 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63600b4b30 0x7f63600d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f63600b4b30 0x7f63600d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47300/0 (socket says 172.21.15.88:47300) 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 -- 172.21.15.88:0/2687056045 learned_addr learned my addr 172.21.15.88:0/2687056045 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f635fe00640 1 --2- 172.21.15.88:0/2687056045 >> v2:172.21.15.132:3300/0 conn(0x7f6360146680 0x7f63600b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 -- 172.21.15.88:0/2687056045 >> v2:172.21.15.132:3300/0 conn(0x7f6360146680 msgr2=0x7f63600b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 --2- 172.21.15.88:0/2687056045 >> v2:172.21.15.132:3300/0 conn(0x7f6360146680 0x7f63600b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 -- 172.21.15.88:0/2687056045 >> v1:172.21.15.132:6789/0 conn(0x7f63600d0e70 legacy=0x7f6360160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.949+0000 7f6365200640 1 -- 172.21.15.88:0/2687056045 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636006dd60 con 0x7f63600b4b30 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.950+0000 7f6365200640 1 --2- 172.21.15.88:0/2687056045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63600b4b30 0x7f63600d12b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6360074250 tx=0x7f635802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f69c7cad5e4d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f635f400640 1 -- 172.21.15.88:0/2687056045 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358036070 con 0x7f63600b4b30 2024-04-26T20:00:41.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f635f400640 1 -- 172.21.15.88:0/2687056045 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6358031600 con 0x7f63600b4b30 2024-04-26T20:00:41.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f635f400640 1 -- 172.21.15.88:0/2687056045 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63580319d0 con 0x7f63600b4b30 2024-04-26T20:00:41.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 -- 172.21.15.88:0/2687056045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63600b4b30 msgr2=0x7f63600d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 --2- 172.21.15.88:0/2687056045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63600b4b30 0x7f63600d12b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6360074250 tx=0x7f635802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 -- 172.21.15.88:0/2687056045 shutdown_connections 2024-04-26T20:00:41.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 --2- 172.21.15.88:0/2687056045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63600b4b30 0x7f63600d12b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 --2- 172.21.15.88:0/2687056045 >> v2:172.21.15.132:3300/0 conn(0x7f6360146680 0x7f63600b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.951+0000 7f6365c00640 1 -- 172.21.15.88:0/2687056045 >> 172.21.15.88:0/2687056045 conn(0x7f636006c740 msgr2=0x7f636006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:41.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.952+0000 7f6365c00640 1 -- 172.21.15.88:0/2687056045 shutdown_connections 2024-04-26T20:00:41.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.952+0000 7f6365c00640 1 -- 172.21.15.88:0/2687056045 wait complete. 2024-04-26T20:00:41.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.953+0000 7f6365c00640 1 Processor -- start 2024-04-26T20:00:41.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.953+0000 7f6365c00640 1 -- start start 2024-04-26T20:00:41.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63600b4b30 0x7f63601435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 0x7f636013d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f636006e600 con 0x7f6360146680 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f636006e320 con 0x7f63600d0e70 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.954+0000 7f6365c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f63600cd6a0 con 0x7f63600b4b30 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47308/0 (socket says 172.21.15.88:47308) 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 -- 172.21.15.88:0/12626885 learned_addr learned my addr 172.21.15.88:0/12626885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6357e00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 0x7f636013d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f635fe00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63600b4b30 0x7f63601435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 -- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63600b4b30 msgr2=0x7f63601435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63600b4b30 0x7f63601435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 -- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 msgr2=0x7f636013d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 0x7f636013d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:41.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 -- 172.21.15.88:0/12626885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6360142230 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6357e00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 0x7f636013d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.955+0000 7f6365200640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6358009f40 tx=0x7f6358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.956+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358036050 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.956+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6358042070 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.956+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358009510 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.956+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6360142510 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.956+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6360142ad0 con 0x7f6360146680 2024-04-26T20:00:41.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.957+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63600693c0 con 0x7f6360146680 2024-04-26T20:00:41.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.958+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6358007c50 con 0x7f6360146680 2024-04-26T20:00:41.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.959+0000 7f635cc00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 0x7f633c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:41.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.959+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f63580acd80 con 0x7f6360146680 2024-04-26T20:00:41.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.959+0000 7f635fe00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 0x7f633c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:41.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.960+0000 7f635fe00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 0x7f633c06ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f63601f6070 tx=0x7f6350008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:41.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:41.964+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6358045360 con 0x7f6360146680 2024-04-26T20:00:42.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.146+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f636015e9f0 con 0x7f6360146680 2024-04-26T20:00:42.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.148+0000 7f635cc00640 1 -- 172.21.15.88:0/12626885 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f635807b080 con 0x7f6360146680 2024-04-26T20:00:42.151 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:42.151 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 msgr2=0x7f633c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 0x7f633c06ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f63601f6070 tx=0x7f6350008040 comp rx=0 tx=0).stop 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 msgr2=0x7f63601f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6358009f40 tx=0x7f6358004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 shutdown_connections 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6360146680 0x7f63601f3ce0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63600d0e70 0x7f636013d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f633c06c920 0x7f633c06ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 --2- 172.21.15.88:0/12626885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f63600b4b30 0x7f63601435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 >> 172.21.15.88:0/12626885 conn(0x7f636006c740 msgr2=0x7f636015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 shutdown_connections 2024-04-26T20:00:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.151+0000 7f6365c00640 1 -- 172.21.15.88:0/12626885 wait complete. 2024-04-26T20:00:42.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:42.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:42.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.369+0000 7f1c31e00640 1 Processor -- start 2024-04-26T20:00:42.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.369+0000 7f1c31e00640 1 -- start start 2024-04-26T20:00:42.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.370+0000 7f1c31e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1c2c160d60 0x7f1c2c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 0x7f1c2c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c0b4d20 con 0x7f1c2c161140 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c06e5f0 con 0x7f1c2c160d60 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c146c30 con 0x7f1c2c146280 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c30a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1c2c160d60 0x7f1c2c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c2be00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1c2c161140 0x7f1c2c06dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47954/0 (socket says 172.21.15.88:47954) 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c2be00640 1 -- 172.21.15.88:0/1083825072 learned_addr learned my addr 172.21.15.88:0/1083825072 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31400640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 0x7f1c2c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c30a00640 1 -- 172.21.15.88:0/1083825072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 msgr2=0x7f1c2c1466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:42.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c30a00640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 0x7f1c2c1466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c30a00640 1 -- 172.21.15.88:0/1083825072 >> v1:172.21.15.88:6789/0 conn(0x7f1c2c161140 legacy=0x7f1c2c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c30a00640 1 -- 172.21.15.88:0/1083825072 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0b5000 con 0x7f1c2c160d60 2024-04-26T20:00:42.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.371+0000 7f1c31400640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 0x7f1c2c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c30a00640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c160d60 0x7f1c2c0b5b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097d0 tx=0x7f1c2002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1765f9b8f2443421 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c2b400640 1 -- 172.21.15.88:0/1083825072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20037070 con 0x7f1c2c160d60 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c2b400640 1 -- 172.21.15.88:0/1083825072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c20032440 con 0x7f1c2c160d60 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c2b400640 1 -- 172.21.15.88:0/1083825072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20032810 con 0x7f1c2c160d60 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 -- 172.21.15.88:0/1083825072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c160d60 msgr2=0x7f1c2c0b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c160d60 0x7f1c2c0b5b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c200097d0 tx=0x7f1c2002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 -- 172.21.15.88:0/1083825072 shutdown_connections 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c146280 0x7f1c2c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 --2- 172.21.15.88:0/1083825072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c160d60 0x7f1c2c0b5b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 -- 172.21.15.88:0/1083825072 >> 172.21.15.88:0/1083825072 conn(0x7f1c2c0b2900 msgr2=0x7f1c2c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:42.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.373+0000 7f1c31e00640 1 -- 172.21.15.88:0/1083825072 shutdown_connections 2024-04-26T20:00:42.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.374+0000 7f1c31e00640 1 -- 172.21.15.88:0/1083825072 wait complete. 2024-04-26T20:00:42.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.375+0000 7f1c31e00640 1 Processor -- start 2024-04-26T20:00:42.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.376+0000 7f1c31e00640 1 -- start start 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c146280 0x7f1c2c0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 0x7f1c2c141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c0cf6f0 con 0x7f1c2c161140 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c0cf410 con 0x7f1c2c146280 2024-04-26T20:00:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c0b6070 con 0x7f1c2c160d60 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47322/0 (socket says 172.21.15.88:47322) 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.377+0000 7f1c31400640 1 -- 172.21.15.88:0/3267420671 learned_addr learned my addr 172.21.15.88:0/3267420671 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c30a00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c146280 0x7f1c2c0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 -- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 msgr2=0x7f1c2c141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c2be00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 0x7f1c2c141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 0x7f1c2c141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 -- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c146280 msgr2=0x7f1c2c0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c146280 0x7f1c2c0cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 -- 172.21.15.88:0/3267420671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c142ac0 con 0x7f1c2c161140 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c2be00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 0x7f1c2c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31400640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cca0 tx=0x7f1c2400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400f680 con 0x7f1c2c161140 2024-04-26T20:00:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2400f840 con 0x7f1c2c161140 2024-04-26T20:00:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400fc30 con 0x7f1c2c161140 2024-04-26T20:00:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c142da0 con 0x7f1c2c161140 2024-04-26T20:00:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.378+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c0d02c0 con 0x7f1c2c161140 2024-04-26T20:00:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.379+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c0693c0 con 0x7f1c2c161140 2024-04-26T20:00:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.380+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c240213f0 con 0x7f1c2c161140 2024-04-26T20:00:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.381+0000 7f1c28c00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 0x7f1c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.381+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f1c2408c8f0 con 0x7f1c2c161140 2024-04-26T20:00:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.382+0000 7f1c30a00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 0x7f1c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.382+0000 7f1c30a00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 0x7f1c0c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.386+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c24014030 con 0x7f1c2c161140 2024-04-26T20:00:42.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.586+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c2c0ce8b0 con 0x7f1c2c161140 2024-04-26T20:00:42.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.587+0000 7f1c28c00640 1 -- 172.21.15.88:0/3267420671 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1c2408c070 con 0x7f1c2c161140 2024-04-26T20:00:42.589 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:42.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 msgr2=0x7f1c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 0x7f1c0c06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1c20034000 tx=0x7f1c2002e000 comp rx=0 tx=0).stop 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 msgr2=0x7f1c2c142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400cca0 tx=0x7f1c2400a650 comp rx=0 tx=0).stop 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 shutdown_connections 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c2c161140 0x7f1c2c142210 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c0c06c8f0 0x7f1c0c06edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1c2c160d60 0x7f1c2c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 --2- 172.21.15.88:0/3267420671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c2c146280 0x7f1c2c0cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 >> 172.21.15.88:0/3267420671 conn(0x7f1c2c0b2900 msgr2=0x7f1c2c073170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:42.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 shutdown_connections 2024-04-26T20:00:42.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.590+0000 7f1c31e00640 1 -- 172.21.15.88:0/3267420671 wait complete. 2024-04-26T20:00:42.611 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:42.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.800+0000 7fab23e00640 1 Processor -- start 2024-04-26T20:00:42.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.800+0000 7fab23e00640 1 -- start start 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.801+0000 7fab23e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab2406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.801+0000 7fab23e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fab24146280 0x7fab241466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.801+0000 7fab23e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab240b4d20 con 0x7fab24160d60 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.801+0000 7fab23e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fab24146c30 con 0x7fab24146280 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.801+0000 7fab23e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fab240b5000 con 0x7fab24161140 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fab24146280 0x7fab241466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fab24146280 0x7fab241466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47338/0 (socket says 172.21.15.88:47338) 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 -- 172.21.15.88:0/1196974503 learned_addr learned my addr 172.21.15.88:0/1196974503 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab22000640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab2406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 -- 172.21.15.88:0/1196974503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 msgr2=0x7fab2406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:42.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab2406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 -- 172.21.15.88:0/1196974503 >> v1:172.21.15.132:6789/0 conn(0x7fab24160d60 legacy=0x7fab240b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:42.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.802+0000 7fab23400640 1 -- 172.21.15.88:0/1196974503 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab240b52e0 con 0x7fab24146280 2024-04-26T20:00:42.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.803+0000 7fab23400640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab241466f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009e50 tx=0x7fab1c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd097c6b9f6c5c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab21600640 1 -- 172.21.15.88:0/1196974503 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c036070 con 0x7fab24146280 2024-04-26T20:00:42.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab21600640 1 -- 172.21.15.88:0/1196974503 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1c031600 con 0x7fab24146280 2024-04-26T20:00:42.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab21600640 1 -- 172.21.15.88:0/1196974503 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0319d0 con 0x7fab24146280 2024-04-26T20:00:42.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 -- 172.21.15.88:0/1196974503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 msgr2=0x7fab241466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab241466f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009e50 tx=0x7fab1c02a710 comp rx=0 tx=0).stop 2024-04-26T20:00:42.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 -- 172.21.15.88:0/1196974503 shutdown_connections 2024-04-26T20:00:42.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab241466f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 --2- 172.21.15.88:0/1196974503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab2406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.804+0000 7fab23e00640 1 -- 172.21.15.88:0/1196974503 >> 172.21.15.88:0/1196974503 conn(0x7fab240b2900 msgr2=0x7fab240b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:42.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.805+0000 7fab23e00640 1 -- 172.21.15.88:0/1196974503 shutdown_connections 2024-04-26T20:00:42.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.805+0000 7fab23e00640 1 -- 172.21.15.88:0/1196974503 wait complete. 2024-04-26T20:00:42.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.806+0000 7fab23e00640 1 Processor -- start 2024-04-26T20:00:42.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.806+0000 7fab23e00640 1 -- start start 2024-04-26T20:00:42.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab24160d60 0x7fab2415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab24159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406e420 con 0x7fab24146280 2024-04-26T20:00:42.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fab240cf6f0 con 0x7fab24160d60 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.807+0000 7fab23e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fab2406e5a0 con 0x7fab24161140 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47342/0 (socket says 172.21.15.88:47342) 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 -- 172.21.15.88:0/3145897835 learned_addr learned my addr 172.21.15.88:0/3145897835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 -- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 msgr2=0x7fab24159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab24159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 -- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab24160d60 msgr2=0x7fab2415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab24160d60 0x7fab2415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 -- 172.21.15.88:0/3145897835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2415a090 con 0x7fab24146280 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab22a00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c400 tx=0x7fab1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.808+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140052d0 con 0x7fab24146280 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.809+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14005490 con 0x7fab24146280 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.809+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14007260 con 0x7fab24146280 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.809+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2415c000 con 0x7fab24146280 2024-04-26T20:00:42.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.809+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241438e0 con 0x7fab24146280 2024-04-26T20:00:42.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.810+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab240693c0 con 0x7fab24146280 2024-04-26T20:00:42.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.811+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab14007400 con 0x7fab24146280 2024-04-26T20:00:42.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.812+0000 7fab1aa00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 0x7fab0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:42.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.812+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fab1408b100 con 0x7fab24146280 2024-04-26T20:00:42.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.812+0000 7fab22000640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 0x7fab0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:42.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.812+0000 7fab22000640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 0x7fab0406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab10006fd0 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:42.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:42.816+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab14059490 con 0x7fab24146280 2024-04-26T20:00:43.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.014+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fab2415c520 con 0x7fab24146280 2024-04-26T20:00:43.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.015+0000 7fab1aa00640 1 -- 172.21.15.88:0/3145897835 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fab14058d90 con 0x7fab24146280 2024-04-26T20:00:43.016 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 msgr2=0x7fab0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 0x7fab0406ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fab10006fd0 tx=0x7fab10008040 comp rx=0 tx=0).stop 2024-04-26T20:00:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 msgr2=0x7fab2415b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c400 tx=0x7fab1400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 shutdown_connections 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fab0406c920 0x7fab0406ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fab24161140 0x7fab24159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab24160d60 0x7fab2415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 --2- 172.21.15.88:0/3145897835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24146280 0x7fab2415b900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.017+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 >> 172.21.15.88:0/3145897835 conn(0x7fab240b2900 msgr2=0x7fab24072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.018+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 shutdown_connections 2024-04-26T20:00:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.018+0000 7fab23e00640 1 -- 172.21.15.88:0/3145897835 wait complete. 2024-04-26T20:00:43.037 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.226+0000 7faf9aa00640 1 Processor -- start 2024-04-26T20:00:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.226+0000 7faf9aa00640 1 -- start start 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.227+0000 7faf9aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf9415d100 0x7faf9406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.227+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 0x7faf9406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.227+0000 7faf9aa00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf940af800 con 0x7faf940add80 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.227+0000 7faf9aa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7faf940ae340 con 0x7faf9415d100 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.227+0000 7faf9aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf940afaa0 con 0x7faf940b5ca0 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf98a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7faf940add80 0x7faf94072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47956/0 (socket says 172.21.15.88:47956) 2024-04-26T20:00:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf98a00640 1 -- 172.21.15.88:0/1028759763 learned_addr learned my addr 172.21.15.88:0/1028759763 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf93e00640 1 --2- 172.21.15.88:0/1028759763 >> v2:172.21.15.132:3300/0 conn(0x7faf9415d100 0x7faf9406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf93400640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 0x7faf9406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2720094568 0 0) 0x7faf940af800 con 0x7faf940add80 2024-04-26T20:00:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.228+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf78003520 con 0x7faf940add80 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2777790192 0 0) 0x7faf78003520 con 0x7faf940add80 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 msgr2=0x7faf9406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 0x7faf9406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 >> v2:172.21.15.132:3300/0 conn(0x7faf9415d100 msgr2=0x7faf9406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 --2- 172.21.15.88:0/1028759763 >> v2:172.21.15.132:3300/0 conn(0x7faf9415d100 0x7faf9406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf940afd80 con 0x7faf940add80 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7faf8c001670 con 0x7faf940add80 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 >> v1:172.21.15.88:6789/0 conn(0x7faf940add80 legacy=0x7faf94072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 0x7faf780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf78003f30 0x7faf78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78004d40 0x7faf780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf940ae340 con 0x7faf78003f30 2024-04-26T20:00:43.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faf940afaa0 con 0x7faf78004d40 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf7800ba10 con 0x7faf940b5ca0 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf78003f30 0x7faf78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 -- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 msgr2=0x7faf780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf940b5ca0 0x7faf780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 -- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78004d40 msgr2=0x7faf780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78004d40 0x7faf780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.229+0000 7faf93400640 1 -- 172.21.15.88:0/1028759763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf78003520 con 0x7faf78003f30 2024-04-26T20:00:43.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.230+0000 7faf93400640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf78003f30 0x7faf78004800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e170 tx=0x7faf78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2a4951dbc0957c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80034020 con 0x7faf78003f30 2024-04-26T20:00:43.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf80030430 con 0x7faf78003f30 2024-04-26T20:00:43.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf92a00640 1 -- 172.21.15.88:0/1028759763 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80009c10 con 0x7faf78003f30 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 -- 172.21.15.88:0/1028759763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf78003f30 msgr2=0x7faf78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 --2- 172.21.15.88:0/1028759763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf78003f30 0x7faf78004800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e170 tx=0x7faf78005830 comp rx=0 tx=0).stop 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf98a00640 1 -- 172.21.15.88:0/1028759763 reap_dead start 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 -- 172.21.15.88:0/1028759763 shutdown_connections 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 -- 172.21.15.88:0/1028759763 >> 172.21.15.88:0/1028759763 conn(0x7faf940b2d50 msgr2=0x7faf940b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 -- 172.21.15.88:0/1028759763 shutdown_connections 2024-04-26T20:00:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.231+0000 7faf9aa00640 1 -- 172.21.15.88:0/1028759763 wait complete. 2024-04-26T20:00:43.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.233+0000 7faf9aa00640 1 Processor -- start 2024-04-26T20:00:43.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.233+0000 7faf9aa00640 1 -- start start 2024-04-26T20:00:43.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78003f30 0x7faf9413d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf9409dc20 0x7faf9409e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf9409ee50 con 0x7faf94158fc0 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faf940b0340 con 0x7faf78003f30 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf9aa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf94160b70 con 0x7faf9409dc20 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf93400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf93400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47368/0 (socket says 172.21.15.88:47368) 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.234+0000 7faf93400640 1 -- 172.21.15.88:0/3368577380 learned_addr learned my addr 172.21.15.88:0/3368577380 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 -- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf9409dc20 msgr2=0x7faf9409e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf9409dc20 0x7faf9409e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 -- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78003f30 msgr2=0x7faf9413d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78003f30 0x7faf9413d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 -- 172.21.15.88:0/3368577380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9415b4f0 con 0x7faf94158fc0 2024-04-26T20:00:43.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.235+0000 7faf93400640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e170 tx=0x7faf80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.236+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80034050 con 0x7faf94158fc0 2024-04-26T20:00:43.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.236+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8003a9f0 con 0x7faf94158fc0 2024-04-26T20:00:43.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.236+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9415b7d0 con 0x7faf94158fc0 2024-04-26T20:00:43.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.236+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80009c10 con 0x7faf94158fc0 2024-04-26T20:00:43.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.236+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9415bd90 con 0x7faf94158fc0 2024-04-26T20:00:43.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.237+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9413da00 con 0x7faf94158fc0 2024-04-26T20:00:43.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.238+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf80037020 con 0x7faf94158fc0 2024-04-26T20:00:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.239+0000 7faf8be00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 0x7faf7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.239+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7faf800ac9f0 con 0x7faf94158fc0 2024-04-26T20:00:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.239+0000 7faf93e00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 0x7faf7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.240+0000 7faf93e00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 0x7faf7406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf84006e80 tx=0x7faf84006dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.245+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf8003f070 con 0x7faf94158fc0 2024-04-26T20:00:43.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.440+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf94069650 con 0x7faf94158fc0 2024-04-26T20:00:43.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.441+0000 7faf8be00640 1 -- 172.21.15.88:0/3368577380 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7faf8004b340 con 0x7faf94158fc0 2024-04-26T20:00:43.444 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 msgr2=0x7faf7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 0x7faf7406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf84006e80 tx=0x7faf84006dd0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 msgr2=0x7faf9415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e170 tx=0x7faf80009750 comp rx=0 tx=0).stop 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 shutdown_connections 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf7406c8f0 0x7faf7406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf9409dc20 0x7faf9409e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf94158fc0 0x7faf9415ac30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 --2- 172.21.15.88:0/3368577380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf78003f30 0x7faf9413d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 >> 172.21.15.88:0/3368577380 conn(0x7faf940b2d50 msgr2=0x7faf94160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.444+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 shutdown_connections 2024-04-26T20:00:43.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.445+0000 7faf9aa00640 1 -- 172.21.15.88:0/3368577380 wait complete. 2024-04-26T20:00:43.464 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:43.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.651+0000 7f71f7200640 1 Processor -- start 2024-04-26T20:00:43.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.651+0000 7f71f7200640 1 -- start start 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71f7200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71f006ce80 0x7f71f01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71f7200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f00b5fb0 con 0x7f71f01598a0 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71f7200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f0160370 con 0x7f71f015f740 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71f7200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f71f0159c80 con 0x7f71f006ce80 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71ef400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71f006ce80 0x7f71f01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71ef400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71f006ce80 0x7f71f01638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47374/0 (socket says 172.21.15.88:47374) 2024-04-26T20:00:43.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71ef400640 1 -- 172.21.15.88:0/3773098544 learned_addr learned my addr 172.21.15.88:0/3773098544 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:43.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.653+0000 7f71efe00640 1 --1- 172.21.15.88:0/3773098544 >> v1:172.21.15.88:6789/0 conn(0x7f71f01598a0 0x7f71f006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47972/0 (socket says 172.21.15.88:47972) 2024-04-26T20:00:43.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.654+0000 7f71ef400640 1 -- 172.21.15.88:0/3773098544 >> v1:172.21.15.132:6789/0 conn(0x7f71f015f740 legacy=0x7f71f015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:43.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.654+0000 7f71ef400640 1 -- 172.21.15.88:0/3773098544 >> v1:172.21.15.88:6789/0 conn(0x7f71f01598a0 legacy=0x7f71f006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.654+0000 7f71ef400640 1 -- 172.21.15.88:0/3773098544 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f0162dd0 con 0x7f71f006ce80 2024-04-26T20:00:43.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.655+0000 7f71ef400640 1 --2- 172.21.15.88:0/3773098544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f006ce80 0x7f71f01638b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394fb8d45ca0e903 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.655+0000 7f71eea00640 1 -- 172.21.15.88:0/3773098544 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc037070 con 0x7f71f006ce80 2024-04-26T20:00:43.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.655+0000 7f71eea00640 1 -- 172.21.15.88:0/3773098544 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc032440 con 0x7f71f006ce80 2024-04-26T20:00:43.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.655+0000 7f71eea00640 1 -- 172.21.15.88:0/3773098544 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc032810 con 0x7f71f006ce80 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 -- 172.21.15.88:0/3773098544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f006ce80 msgr2=0x7f71f01638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 --2- 172.21.15.88:0/3773098544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f006ce80 0x7f71f01638b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a210 comp rx=0 tx=0).stop 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 -- 172.21.15.88:0/3773098544 shutdown_connections 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 --2- 172.21.15.88:0/3773098544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f006ce80 0x7f71f01638b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 -- 172.21.15.88:0/3773098544 >> 172.21.15.88:0/3773098544 conn(0x7f71f00b2d50 msgr2=0x7f71f00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 -- 172.21.15.88:0/3773098544 shutdown_connections 2024-04-26T20:00:43.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.656+0000 7f71f7200640 1 -- 172.21.15.88:0/3773098544 wait complete. 2024-04-26T20:00:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.658+0000 7f71f7200640 1 Processor -- start 2024-04-26T20:00:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.658+0000 7f71f7200640 1 -- start start 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f006ce80 0x7f71f00cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71f015f740 0x7f71f01464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0160650 con 0x7f71f01598a0 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71f0163390 con 0x7f71f006ce80 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71f7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71f00add80 con 0x7f71f015f740 2024-04-26T20:00:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47380/0 (socket says 172.21.15.88:47380) 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.659+0000 7f71ef400640 1 -- 172.21.15.88:0/4044355252 learned_addr learned my addr 172.21.15.88:0/4044355252 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 -- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71f015f740 msgr2=0x7f71f01464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71f015f740 0x7f71f01464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 -- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f006ce80 msgr2=0x7f71f00cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f006ce80 0x7f71f00cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 -- 172.21.15.88:0/4044355252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f00cf320 con 0x7f71f01598a0 2024-04-26T20:00:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71ef400640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc041020 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc030970 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc037040 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f00cf600 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.660+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f00cfbc0 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.661+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f0069540 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.663+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71dc044020 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.663+0000 7f71e7e00640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.664+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f71dc0ac620 con 0x7f71f01598a0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.664+0000 7f71efe00640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.664+0000 7f71efe00640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 0x7f71d006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000f60 tx=0x7f71e0007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:43.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.668+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71dc04b030 con 0x7f71f01598a0 2024-04-26T20:00:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.871+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f71f00cd470 con 0x7f71f01598a0 2024-04-26T20:00:43.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.872+0000 7f71e7e00640 1 -- 172.21.15.88:0/4044355252 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f71f00cd470 con 0x7f71f01598a0 2024-04-26T20:00:43.874 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 msgr2=0x7f71d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 0x7f71d006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000f60 tx=0x7f71e0007a00 comp rx=0 tx=0).stop 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 msgr2=0x7f71f01443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 shutdown_connections 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71f015f740 0x7f71f01464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71f01598a0 0x7f71f01443f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 --2- 172.21.15.88:0/4044355252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f006ce80 0x7f71f00cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:43.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 >> 172.21.15.88:0/4044355252 conn(0x7f71f00b2d50 msgr2=0x7f71f006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:43.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 shutdown_connections 2024-04-26T20:00:43.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:43.875+0000 7f71f7200640 1 -- 172.21.15.88:0/4044355252 wait complete. 2024-04-26T20:00:43.894 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-04-26T20:00:43.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:43.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-26T20:00:43.968 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:43.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:00:43.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:43.968 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T20:00:44.013 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:44.014 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:44.069 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:44.069 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:44.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:44.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.320+0000 7fa77c800640 1 Processor -- start 2024-04-26T20:00:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.320+0000 7fa77c800640 1 -- start start 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.321+0000 7fa77c800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa77406ce80 0x7fa7740add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.321+0000 7fa77c800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7740af9a0 con 0x7fa7740cd650 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.321+0000 7fa77c800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7740afc80 con 0x7fa77415cdf0 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.321+0000 7fa77c800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa7740ae2c0 con 0x7fa77406ce80 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa77a800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa7740cd650 0x7fa7740b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47974/0 (socket says 172.21.15.88:47974) 2024-04-26T20:00:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa77a800640 1 -- 172.21.15.88:0/896905932 learned_addr learned my addr 172.21.15.88:0/896905932 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:44.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa779400640 1 --2- 172.21.15.88:0/896905932 >> v2:172.21.15.132:3300/0 conn(0x7fa77406ce80 0x7fa7740add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892505255 0 0) 0x7fa7740af9a0 con 0x7fa7740cd650 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa75c003520 con 0x7fa7740cd650 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2427648422 0 0) 0x7fa75c003520 con 0x7fa7740cd650 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 >> v2:172.21.15.132:3300/0 conn(0x7fa77406ce80 msgr2=0x7fa7740add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 --2- 172.21.15.88:0/896905932 >> v2:172.21.15.132:3300/0 conn(0x7fa77406ce80 0x7fa7740add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.322+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 >> v1:172.21.15.132:6789/0 conn(0x7fa77415cdf0 legacy=0x7fa77406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7740aff60 con 0x7fa7740cd650 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa7700019f0 con 0x7fa7740cd650 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 >> v1:172.21.15.88:6789/0 conn(0x7fa7740cd650 legacy=0x7fa7740b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0039b0 0x7fa75c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa75c004800 0x7fa75c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 0x7fa75c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c00c030 con 0x7fa75c0051c0 2024-04-26T20:00:44.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c00c1e0 con 0x7fa75c004800 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c00c3e0 con 0x7fa75c0039b0 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 0x7fa75c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 -- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0039b0 msgr2=0x7fa75c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0039b0 0x7fa75c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 -- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa75c004800 msgr2=0x7fa75c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa75c004800 0x7fa75c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.323+0000 7fa77a800640 1 -- 172.21.15.88:0/896905932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c003520 con 0x7fa75c0051c0 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.324+0000 7fa77a800640 1 -- 172.21.15.88:0/896905932 reap_dead start 2024-04-26T20:00:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.324+0000 7fa77a800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 0x7fa75c00b9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa77000a810 tx=0x7fa75c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8f84759c451c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77000a460 con 0x7fa75c0051c0 2024-04-26T20:00:44.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77002f070 con 0x7fa75c0051c0 2024-04-26T20:00:44.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa778a00640 1 -- 172.21.15.88:0/896905932 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77002bc90 con 0x7fa75c0051c0 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 -- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 msgr2=0x7fa75c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 0x7fa75c00b9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa77000a810 tx=0x7fa75c005830 comp rx=0 tx=0).stop 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 -- 172.21.15.88:0/896905932 shutdown_connections 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 --2- 172.21.15.88:0/896905932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa75c0051c0 0x7fa75c00b9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 -- 172.21.15.88:0/896905932 >> 172.21.15.88:0/896905932 conn(0x7fa7740b2d50 msgr2=0x7fa7740b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 -- 172.21.15.88:0/896905932 shutdown_connections 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.325+0000 7fa77c800640 1 -- 172.21.15.88:0/896905932 wait complete. 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.327+0000 7fa77c800640 1 Processor -- start 2024-04-26T20:00:44.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.327+0000 7fa77c800640 1 -- start start 2024-04-26T20:00:44.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 0x7fa77414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 0x7fa77409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa77415d1d0 con 0x7fa77415dd40 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa7740b05b0 con 0x7fa77409ba50 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa77415d350 con 0x7fa75c0051c0 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47404/0 (socket says 172.21.15.88:47404) 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.328+0000 7fa77a800640 1 -- 172.21.15.88:0/2467506081 learned_addr learned my addr 172.21.15.88:0/2467506081 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa779e00640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 0x7fa77414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa779400640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 0x7fa77409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 -- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 msgr2=0x7fa77414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 0x7fa77414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 -- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 msgr2=0x7fa77409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 0x7fa77409bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 -- 172.21.15.88:0/2467506081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77415e700 con 0x7fa77415dd40 2024-04-26T20:00:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa779400640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 0x7fa77409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa779e00640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 0x7fa77414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa77a800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa77000a030 tx=0x7fa7740b0130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77000a460 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77002bc70 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.329+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77003f660 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.330+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77409c4d0 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.330+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77409ca90 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.331+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa774069540 con 0x7fa77415dd40 2024-04-26T20:00:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.332+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa770031030 con 0x7fa77415dd40 2024-04-26T20:00:44.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.332+0000 7fa76e000640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.333+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fa7700aadf0 con 0x7fa77415dd40 2024-04-26T20:00:44.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.333+0000 7fa779e00640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.333+0000 7fa779e00640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 0x7fa75806ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa768001660 tx=0x7fa768009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.337+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa770046030 con 0x7fa77415dd40 2024-04-26T20:00:44.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.522+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa77406c9c0 con 0x7fa77415dd40 2024-04-26T20:00:44.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.524+0000 7fa76e000640 1 -- 172.21.15.88:0/2467506081 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7fa77415e700 con 0x7fa77415dd40 2024-04-26T20:00:44.527 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:44.527 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 msgr2=0x7fa75806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 0x7fa75806ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa768001660 tx=0x7fa768009040 comp rx=0 tx=0).stop 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 msgr2=0x7fa77415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa77000a030 tx=0x7fa7740b0130 comp rx=0 tx=0).stop 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 shutdown_connections 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa77415dd40 0x7fa77415e1c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa77409ba50 0x7fa77409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa75806c920 0x7fa75806ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 --2- 172.21.15.88:0/2467506081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa75c0051c0 0x7fa77414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 >> 172.21.15.88:0/2467506081 conn(0x7fa7740b2d50 msgr2=0x7fa77415c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:44.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.527+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 shutdown_connections 2024-04-26T20:00:44.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.528+0000 7fa77c800640 1 -- 172.21.15.88:0/2467506081 wait complete. 2024-04-26T20:00:44.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:44.546 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:44.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.739+0000 7f4e7ca00640 1 Processor -- start 2024-04-26T20:00:44.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.739+0000 7f4e7ca00640 1 -- start start 2024-04-26T20:00:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.741+0000 7f4e7ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4e78160d60 0x7f4e780b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.741+0000 7f4e7ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 0x7f4e781466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.741+0000 7f4e7ca00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e780b4d20 con 0x7f4e78161140 2024-04-26T20:00:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.741+0000 7f4e7ca00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7806e5f0 con 0x7f4e78160d60 2024-04-26T20:00:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.741+0000 7f4e7ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4e78146c30 con 0x7f4e78146280 2024-04-26T20:00:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e77e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 0x7f4e781466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e76a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4e78161140 0x7f4e7806dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:52994/0 (socket says 172.21.15.88:52994) 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e76a00640 1 -- 172.21.15.88:0/781494187 learned_addr learned my addr 172.21.15.88:0/781494187 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e77400640 1 --2- 172.21.15.88:0/781494187 >> v2:172.21.15.132:3300/0 conn(0x7f4e78160d60 0x7f4e780b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e77e00640 1 -- 172.21.15.88:0/781494187 >> v2:172.21.15.132:3300/0 conn(0x7f4e78160d60 msgr2=0x7f4e780b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.742+0000 7f4e77e00640 1 --2- 172.21.15.88:0/781494187 >> v2:172.21.15.132:3300/0 conn(0x7f4e78160d60 0x7f4e780b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.743+0000 7f4e77e00640 1 -- 172.21.15.88:0/781494187 >> v1:172.21.15.132:6789/0 conn(0x7f4e78161140 legacy=0x7f4e7806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.743+0000 7f4e77e00640 1 -- 172.21.15.88:0/781494187 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e780b5000 con 0x7f4e78146280 2024-04-26T20:00:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.744+0000 7f4e77e00640 1 --2- 172.21.15.88:0/781494187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 0x7f4e781466f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097d0 tx=0x7f4e6802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0390e83830f42c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.745+0000 7f4e76000640 1 -- 172.21.15.88:0/781494187 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68037070 con 0x7f4e78146280 2024-04-26T20:00:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.745+0000 7f4e76000640 1 -- 172.21.15.88:0/781494187 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e68032440 con 0x7f4e78146280 2024-04-26T20:00:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.745+0000 7f4e76000640 1 -- 172.21.15.88:0/781494187 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68032810 con 0x7f4e78146280 2024-04-26T20:00:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.745+0000 7f4e7ca00640 1 -- 172.21.15.88:0/781494187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 msgr2=0x7f4e781466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.745+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/781494187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 0x7f4e781466f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097d0 tx=0x7f4e6802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 -- 172.21.15.88:0/781494187 shutdown_connections 2024-04-26T20:00:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/781494187 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78146280 0x7f4e781466f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/781494187 >> v2:172.21.15.132:3300/0 conn(0x7f4e78160d60 0x7f4e780b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 -- 172.21.15.88:0/781494187 >> 172.21.15.88:0/781494187 conn(0x7f4e780b2900 msgr2=0x7f4e780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 -- 172.21.15.88:0/781494187 shutdown_connections 2024-04-26T20:00:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.746+0000 7f4e7ca00640 1 -- 172.21.15.88:0/781494187 wait complete. 2024-04-26T20:00:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.748+0000 7f4e7ca00640 1 Processor -- start 2024-04-26T20:00:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.748+0000 7f4e7ca00640 1 -- start start 2024-04-26T20:00:44.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.749+0000 7f4e7ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e78146280 0x7f4e7813e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e7ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 0x7f4e78073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e7ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e7ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e780cf6f0 con 0x7f4e78161140 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e7ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4e780cf410 con 0x7f4e78146280 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e7ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4e780b6070 con 0x7f4e78160d60 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e77e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e77e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47408/0 (socket says 172.21.15.88:47408) 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e77e00640 1 -- 172.21.15.88:0/871078053 learned_addr learned my addr 172.21.15.88:0/871078053 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e77400640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e78146280 0x7f4e7813e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.750+0000 7f4e76a00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 0x7f4e78073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 -- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 msgr2=0x7f4e78073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 0x7f4e78073310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 -- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e78146280 msgr2=0x7f4e7813e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e78146280 0x7f4e7813e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 -- 172.21.15.88:0/871078053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7815b2e0 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e76a00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 0x7f4e78073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e77e00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e680098f0 tx=0x7f4e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68041020 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6803d640 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68037040 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.751+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7815b5c0 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.752+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7813e790 con 0x7f4e78161140 2024-04-26T20:00:44.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.753+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e780693c0 con 0x7f4e78161140 2024-04-26T20:00:44.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.754+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e68044020 con 0x7f4e78161140 2024-04-26T20:00:44.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.754+0000 7f4e6f400640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 0x7f4e5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:44.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.755+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f4e680ac2a0 con 0x7f4e78161140 2024-04-26T20:00:44.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.755+0000 7f4e77400640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 0x7f4e5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:44.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.755+0000 7f4e77400640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 0x7f4e5406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e700113e0 tx=0x7f4e7000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:44.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.759+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e6804b030 con 0x7f4e78161140 2024-04-26T20:00:44.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.963+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e78076640 con 0x7f4e78161140 2024-04-26T20:00:44.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.964+0000 7f4e6f400640 1 -- 172.21.15.88:0/871078053 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4e6807a5b0 con 0x7f4e78161140 2024-04-26T20:00:44.965 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 msgr2=0x7f4e5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 0x7f4e5406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4e700113e0 tx=0x7f4e7000b040 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 msgr2=0x7f4e78073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e680098f0 tx=0x7f4e680046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 shutdown_connections 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e78161140 0x7f4e78073850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4e5406c920 0x7f4e5406ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4e78160d60 0x7f4e78073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 --2- 172.21.15.88:0/871078053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e78146280 0x7f4e7813e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 >> 172.21.15.88:0/871078053 conn(0x7f4e780b2900 msgr2=0x7f4e7806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 shutdown_connections 2024-04-26T20:00:44.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:44.967+0000 7f4e7ca00640 1 -- 172.21.15.88:0/871078053 wait complete. 2024-04-26T20:00:44.988 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:45.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.186+0000 7f71f2800640 1 Processor -- start 2024-04-26T20:00:45.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.186+0000 7f71f2800640 1 -- start start 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.188+0000 7f71f2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f71ec06c350 0x7f71ec06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.188+0000 7f71f2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 0x7f71ec162ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.188+0000 7f71f2800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71ec06cc70 con 0x7f71ec0b6c60 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.188+0000 7f71f2800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f71ec073c80 con 0x7f71ec06c350 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.188+0000 7f71f2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec073f60 con 0x7f71ec0b67e0 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71f1e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f71ec0b6c60 0x7f71ec1630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47976/0 (socket says 172.21.15.88:47976) 2024-04-26T20:00:45.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71f1e00640 1 -- 172.21.15.88:0/775058209 learned_addr learned my addr 172.21.15.88:0/775058209 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71f0a00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 0x7f71ec162ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71f1400640 1 --2- 172.21.15.88:0/775058209 >> v2:172.21.15.132:3300/0 conn(0x7f71ec06c350 0x7f71ec06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175337279 0 0) 0x7f71ec06cc70 con 0x7f71ec0b6c60 2024-04-26T20:00:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71d0003520 con 0x7f71ec0b6c60 2024-04-26T20:00:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4147860805 0 0) 0x7f71d0003520 con 0x7f71ec0b6c60 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 msgr2=0x7f71ec162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 0x7f71ec162ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 >> v2:172.21.15.132:3300/0 conn(0x7f71ec06c350 msgr2=0x7f71ec06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 --2- 172.21.15.88:0/775058209 >> v2:172.21.15.132:3300/0 conn(0x7f71ec06c350 0x7f71ec06c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec0b4ba0 con 0x7f71ec0b6c60 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f71e4001670 con 0x7f71ec0b6c60 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71f1400640 1 --2- 172.21.15.88:0/775058209 >> v2:172.21.15.132:3300/0 conn(0x7f71ec06c350 0x7f71ec06c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 >> v1:172.21.15.88:6789/0 conn(0x7f71ec0b6c60 legacy=0x7f71ec1630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0003a20 0x7f71d0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.189+0000 7f71ebe00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71d0004800 0x7f71d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71ebe00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 0x7f71d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec073c80 con 0x7f71d0004800 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec073f60 con 0x7f71d0003a20 2024-04-26T20:00:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71d000ba40 con 0x7f71ec0b67e0 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71d0004800 0x7f71d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f1400640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0003a20 0x7f71d0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 -- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 msgr2=0x7f71d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec0b67e0 0x7f71d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 -- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0003a20 msgr2=0x7f71d0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0003a20 0x7f71d0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f0a00640 1 -- 172.21.15.88:0/775058209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d0003520 con 0x7f71d0004800 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.190+0000 7f71f1400640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0003a20 0x7f71d0003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.191+0000 7f71f0a00640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71d0004800 0x7f71d0004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009fc0 tx=0x7f71d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aed17971b9bb591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.191+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8009c10 con 0x7f71d0004800 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.191+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8036070 con 0x7f71d0004800 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.191+0000 7f71ebe00640 1 -- 172.21.15.88:0/775058209 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d802d070 con 0x7f71d0004800 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 -- 172.21.15.88:0/775058209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71d0004800 msgr2=0x7f71d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 --2- 172.21.15.88:0/775058209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71d0004800 0x7f71d0004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009fc0 tx=0x7f71d0005830 comp rx=0 tx=0).stop 2024-04-26T20:00:45.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f1e00640 1 -- 172.21.15.88:0/775058209 reap_dead start 2024-04-26T20:00:45.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 -- 172.21.15.88:0/775058209 shutdown_connections 2024-04-26T20:00:45.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 -- 172.21.15.88:0/775058209 >> 172.21.15.88:0/775058209 conn(0x7f71ec0b38a0 msgr2=0x7f71ec0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:45.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 -- 172.21.15.88:0/775058209 shutdown_connections 2024-04-26T20:00:45.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.192+0000 7f71f2800640 1 -- 172.21.15.88:0/775058209 wait complete. 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.194+0000 7f71f2800640 1 Processor -- start 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.194+0000 7f71f2800640 1 -- start start 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0004800 0x7f71ec145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec15eb10 0x7f71ec1e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec0ae8c0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec073970 con 0x7f71d0004800 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec0aea40 con 0x7f71ec15eb10 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47424/0 (socket says 172.21.15.88:47424) 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 -- 172.21.15.88:0/2475782126 learned_addr learned my addr 172.21.15.88:0/2475782126 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 -- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec15eb10 msgr2=0x7f71ec1e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec15eb10 0x7f71ec1e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1e00640 1 -- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0004800 msgr2=0x7f71ec145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.195+0000 7f71f1400640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0004800 0x7f71ec145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71f1e00640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0004800 0x7f71ec145980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71f1e00640 1 -- 172.21.15.88:0/2475782126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec144ee0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71f1e00640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71ec075fa0 tx=0x7f71e400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004bf0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e4004d90 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4007270 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec1e92f0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.196+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec145180 con 0x7f71ec1e8b70 2024-04-26T20:00:45.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.197+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec0b55f0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.198+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71e4007410 con 0x7f71ec1e8b70 2024-04-26T20:00:45.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.199+0000 7f71e9600640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 0x7f71cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.199+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f71e408ac30 con 0x7f71ec1e8b70 2024-04-26T20:00:45.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.199+0000 7f71f1400640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 0x7f71cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.200+0000 7f71f1400640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 0x7f71cc06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71dc006fd0 tx=0x7f71dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.203+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71ec0b55f0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.390+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71ec069ad0 con 0x7f71ec1e8b70 2024-04-26T20:00:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.391+0000 7f71e9600640 1 -- 172.21.15.88:0/2475782126 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f71e401a030 con 0x7f71ec1e8b70 2024-04-26T20:00:45.393 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.393+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 msgr2=0x7f71cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.393+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 0x7f71cc06ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71dc006fd0 tx=0x7f71dc008040 comp rx=0 tx=0).stop 2024-04-26T20:00:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.393+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 msgr2=0x7f71ec143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.393+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71ec075fa0 tx=0x7f71e400c7a0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 shutdown_connections 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71ec1e8b70 0x7f71ec143400 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71cc06c920 0x7f71cc06ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71ec15eb10 0x7f71ec1e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 --2- 172.21.15.88:0/2475782126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71d0004800 0x7f71ec145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 >> 172.21.15.88:0/2475782126 conn(0x7f71ec0b38a0 msgr2=0x7f71ec0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 shutdown_connections 2024-04-26T20:00:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.394+0000 7f71f2800640 1 -- 172.21.15.88:0/2475782126 wait complete. 2024-04-26T20:00:45.414 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.610+0000 7f673d000640 1 Processor -- start 2024-04-26T20:00:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.611+0000 7f673d000640 1 -- start start 2024-04-26T20:00:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.612+0000 7f673d000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67380b6210 0x7f673816a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.612+0000 7f673d000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380b6710 con 0x7f67380d3110 2024-04-26T20:00:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.612+0000 7f673d000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380d3600 con 0x7f67381632e0 2024-04-26T20:00:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.612+0000 7f673d000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f67380d38e0 con 0x7f67380b6210 2024-04-26T20:00:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.612+0000 7f6737e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67380b6210 0x7f673816a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f672ea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f67380d3110 0x7f6738162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47986/0 (socket says 172.21.15.88:47986) 2024-04-26T20:00:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f672ea00640 1 -- 172.21.15.88:0/1940767029 learned_addr learned my addr 172.21.15.88:0/1940767029 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f6737e00640 1 --2- 172.21.15.88:0/1940767029 >> v2:172.21.15.88:3300/0 conn(0x7f67380b6210 0x7f673816a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47432/0 (socket says 172.21.15.88:47432) 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f6737e00640 1 -- 172.21.15.88:0/1940767029 >> v1:172.21.15.132:6789/0 conn(0x7f67381632e0 legacy=0x7f67381636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f6737e00640 1 -- 172.21.15.88:0/1940767029 >> v1:172.21.15.88:6789/0 conn(0x7f67380d3110 legacy=0x7f6738162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f6737e00640 1 -- 172.21.15.88:0/1940767029 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380d3bc0 con 0x7f67380b6210 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.613+0000 7f6736a00640 1 -- 172.21.15.88:0/1940767029 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 706369954 0 0) 0x7f67380b6710 con 0x7f67380d3110 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.614+0000 7f6737e00640 1 --2- 172.21.15.88:0/1940767029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f673816a3a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7623cca47b0dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.614+0000 7f6736a00640 1 -- 172.21.15.88:0/1940767029 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730031600 con 0x7f67380b6210 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.614+0000 7f6736a00640 1 -- 172.21.15.88:0/1940767029 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67300317a0 con 0x7f67380b6210 2024-04-26T20:00:45.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.614+0000 7f6736a00640 1 -- 172.21.15.88:0/1940767029 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730031b70 con 0x7f67380b6210 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 -- 172.21.15.88:0/1940767029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 msgr2=0x7f673816a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 --2- 172.21.15.88:0/1940767029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f673816a3a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673002a610 comp rx=0 tx=0).stop 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 -- 172.21.15.88:0/1940767029 shutdown_connections 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 --2- 172.21.15.88:0/1940767029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f673816a3a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 -- 172.21.15.88:0/1940767029 >> 172.21.15.88:0/1940767029 conn(0x7f673809e2f0 msgr2=0x7f67380d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 -- 172.21.15.88:0/1940767029 shutdown_connections 2024-04-26T20:00:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.615+0000 7f673d000640 1 -- 172.21.15.88:0/1940767029 wait complete. 2024-04-26T20:00:45.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.617+0000 7f673d000640 1 Processor -- start 2024-04-26T20:00:45.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.617+0000 7f673d000640 1 -- start start 2024-04-26T20:00:45.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 0x7f6738150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67381632e0 0x7f67381616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67380b3260 con 0x7f67380b6210 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6738163dd0 con 0x7f67380d3110 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f673d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f67380b33e0 con 0x7f67381632e0 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47446/0 (socket says 172.21.15.88:47446) 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 -- 172.21.15.88:0/4265388789 learned_addr learned my addr 172.21.15.88:0/4265388789 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f672ea00640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 0x7f6738150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 -- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67381632e0 msgr2=0x7f67381616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67381632e0 0x7f67381616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 -- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 msgr2=0x7f6738150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 0x7f6738150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f6737400640 1 -- 172.21.15.88:0/4265388789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381f9c00 con 0x7f67380b6210 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.618+0000 7f672ea00640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 0x7f6738150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f6737400640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f672800b420 tx=0x7f672800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728011830 con 0x7f67380b6210 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6728016070 con 0x7f67380b6210 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6738160340 con 0x7f67380b6210 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728011c10 con 0x7f67380b6210 2024-04-26T20:00:45.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.619+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6738160be0 con 0x7f67380b6210 2024-04-26T20:00:45.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.620+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67380693c0 con 0x7f67380b6210 2024-04-26T20:00:45.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.621+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f672801b460 con 0x7f67380b6210 2024-04-26T20:00:45.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.622+0000 7f672fe00640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 0x7f671406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:45.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.622+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f672808b700 con 0x7f67380b6210 2024-04-26T20:00:45.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.622+0000 7f672ea00640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 0x7f671406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:45.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.623+0000 7f672ea00640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 0x7f671406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67380b2f10 tx=0x7f6720009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:45.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.626+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6728020020 con 0x7f67380b6210 2024-04-26T20:00:45.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.810+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67380d2c50 con 0x7f67380b6210 2024-04-26T20:00:45.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.811+0000 7f672fe00640 1 -- 172.21.15.88:0/4265388789 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f6728059a00 con 0x7f67380b6210 2024-04-26T20:00:45.813 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 msgr2=0x7f671406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 0x7f671406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67380b2f10 tx=0x7f6720009040 comp rx=0 tx=0).stop 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 msgr2=0x7f67381f96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f672800b420 tx=0x7f672800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 shutdown_connections 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f671406c8f0 0x7f671406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67381632e0 0x7f67381616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67380d3110 0x7f6738150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 --2- 172.21.15.88:0/4265388789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67380b6210 0x7f67381f96c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:45.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 >> 172.21.15.88:0/4265388789 conn(0x7f673809e2f0 msgr2=0x7f67380b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:45.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 shutdown_connections 2024-04-26T20:00:45.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:45.814+0000 7f673d000640 1 -- 172.21.15.88:0/4265388789 wait complete. 2024-04-26T20:00:45.835 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:46.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.030+0000 7fd55ca00640 1 Processor -- start 2024-04-26T20:00:46.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.031+0000 7fd55ca00640 1 -- start start 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 0x7fd5580d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd558162b00 0x7fd558162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd558163480 0x7fd5580cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd558163910 con 0x7fd558162b00 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd5580d3f50 con 0x7fd558163480 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd55ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd5580d36b0 con 0x7fd55815f1b0 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd556a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd558162b00 0x7fd558162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd556a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd558162b00 0x7fd558162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47456/0 (socket says 172.21.15.88:47456) 2024-04-26T20:00:46.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.032+0000 7fd556a00640 1 -- 172.21.15.88:0/3361855629 learned_addr learned my addr 172.21.15.88:0/3361855629 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd557e00640 1 --2- 172.21.15.88:0/3361855629 >> v2:172.21.15.132:3300/0 conn(0x7fd558163480 0x7fd5580cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd557400640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 0x7fd5580d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd556a00640 1 -- 172.21.15.88:0/3361855629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 msgr2=0x7fd5580d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd556a00640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 0x7fd5580d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd556a00640 1 -- 172.21.15.88:0/3361855629 >> v2:172.21.15.132:3300/0 conn(0x7fd558163480 msgr2=0x7fd5580cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd556a00640 1 --2- 172.21.15.88:0/3361855629 >> v2:172.21.15.132:3300/0 conn(0x7fd558163480 0x7fd5580cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd556a00640 1 -- 172.21.15.88:0/3361855629 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5580d3990 con 0x7fd558162b00 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.033+0000 7fd557400640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 0x7fd5580d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.034+0000 7fd556a00640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558162b00 0x7fd558162f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5580c8770 tx=0x7fd54002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fcfeb07d76a925 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.034+0000 7fd556000640 1 -- 172.21.15.88:0/3361855629 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540036070 con 0x7fd558162b00 2024-04-26T20:00:46.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.034+0000 7fd556000640 1 -- 172.21.15.88:0/3361855629 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd540031600 con 0x7fd558162b00 2024-04-26T20:00:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.034+0000 7fd556000640 1 -- 172.21.15.88:0/3361855629 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400319d0 con 0x7fd558162b00 2024-04-26T20:00:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 -- 172.21.15.88:0/3361855629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558162b00 msgr2=0x7fd558162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558162b00 0x7fd558162f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5580c8770 tx=0x7fd54002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 -- 172.21.15.88:0/3361855629 shutdown_connections 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3361855629 >> v2:172.21.15.132:3300/0 conn(0x7fd558163480 0x7fd5580cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558162b00 0x7fd558162f40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3361855629 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55815f1b0 0x7fd5580d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 -- 172.21.15.88:0/3361855629 >> 172.21.15.88:0/3361855629 conn(0x7fd55809e230 msgr2=0x7fd5580d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 -- 172.21.15.88:0/3361855629 shutdown_connections 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.035+0000 7fd55ca00640 1 -- 172.21.15.88:0/3361855629 wait complete. 2024-04-26T20:00:46.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.037+0000 7fd55ca00640 1 Processor -- start 2024-04-26T20:00:46.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.037+0000 7fd55ca00640 1 -- start start 2024-04-26T20:00:46.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.038+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.038+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd558162b00 0x7fd5580caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd558163480 0x7fd55806dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd55ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5580d33a0 con 0x7fd55815f1b0 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd55ca00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd5580b7d90 con 0x7fd558163480 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd55ca00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd5580d0010 con 0x7fd558162b00 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47468/0 (socket says 172.21.15.88:47468) 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 -- 172.21.15.88:0/1506051340 learned_addr learned my addr 172.21.15.88:0/1506051340 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd556a00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd558162b00 0x7fd5580caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557e00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd558163480 0x7fd55806dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 -- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd558162b00 msgr2=0x7fd5580caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd558162b00 0x7fd5580caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 -- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd558163480 msgr2=0x7fd55806dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd558163480 0x7fd55806dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.039+0000 7fd557400640 1 -- 172.21.15.88:0/1506051340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5580cb410 con 0x7fd55815f1b0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd557400640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c7c0 tx=0x7fd54800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548015070 con 0x7fd55815f1b0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd548007a30 con 0x7fd55815f1b0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5580ce100 con 0x7fd55815f1b0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548007e00 con 0x7fd55815f1b0 2024-04-26T20:00:46.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.040+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5580ce6c0 con 0x7fd55815f1b0 2024-04-26T20:00:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.041+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5580693c0 con 0x7fd55815f1b0 2024-04-26T20:00:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.042+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd548018070 con 0x7fd55815f1b0 2024-04-26T20:00:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.043+0000 7fd54f400640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 0x7fd53406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.043+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fd54808bac0 con 0x7fd55815f1b0 2024-04-26T20:00:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.043+0000 7fd556a00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 0x7fd53406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.044+0000 7fd556a00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 0x7fd53406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd540033000 tx=0x7fd540009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.047+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd548059e60 con 0x7fd55815f1b0 2024-04-26T20:00:46.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.259+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd55808d090 con 0x7fd55815f1b0 2024-04-26T20:00:46.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.260+0000 7fd54f400640 1 -- 172.21.15.88:0/1506051340 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd548059760 con 0x7fd55815f1b0 2024-04-26T20:00:46.262 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 msgr2=0x7fd53406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 0x7fd53406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd540033000 tx=0x7fd540009000 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 msgr2=0x7fd5581531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c7c0 tx=0x7fd54800cc80 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 shutdown_connections 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd558163480 0x7fd55806dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53406c8f0 0x7fd53406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd558162b00 0x7fd5580caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1506051340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55815f1b0 0x7fd5581531f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 >> 172.21.15.88:0/1506051340 conn(0x7fd55809e230 msgr2=0x7fd5580ba650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 shutdown_connections 2024-04-26T20:00:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.263+0000 7fd55ca00640 1 -- 172.21.15.88:0/1506051340 wait complete. 2024-04-26T20:00:46.282 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:00:46.282 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:00:46.340 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:46.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.542+0000 7ff04ee00640 1 Processor -- start 2024-04-26T20:00:46.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.542+0000 7ff04ee00640 1 -- start start 2024-04-26T20:00:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff04815cdf0 0x7ff0480add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff04806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ee00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0480af9a0 con 0x7ff0480b4f50 2024-04-26T20:00:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff0480ae2c0 con 0x7ff04815cdf0 2024-04-26T20:00:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff0480b05b0 con 0x7ff0480cd650 2024-04-26T20:00:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ce00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff0480b4f50 0x7ff048072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47988/0 (socket says 172.21.15.88:47988) 2024-04-26T20:00:46.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.544+0000 7ff04ce00640 1 -- 172.21.15.88:0/2531640231 learned_addr learned my addr 172.21.15.88:0/2531640231 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:46.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437496853 0 0) 0x7ff0480af9a0 con 0x7ff0480b4f50 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff02c003520 con 0x7ff0480b4f50 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff047e00640 1 --2- 172.21.15.88:0/2531640231 >> v2:172.21.15.132:3300/0 conn(0x7ff04815cdf0 0x7ff0480add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff047400640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff04806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2554227620 0 0) 0x7ff02c003520 con 0x7ff0480b4f50 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 msgr2=0x7ff04806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff04806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 >> v2:172.21.15.132:3300/0 conn(0x7ff04815cdf0 msgr2=0x7ff0480add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 --2- 172.21.15.88:0/2531640231 >> v2:172.21.15.132:3300/0 conn(0x7ff04815cdf0 0x7ff0480add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0480afc80 con 0x7ff0480b4f50 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7ff0400019f0 con 0x7ff0480b4f50 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 >> v1:172.21.15.88:6789/0 conn(0x7ff0480b4f50 legacy=0x7ff048072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff02c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff02c003fe0 0x7ff02c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c004d40 0x7ff02c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0480b05b0 con 0x7ff02c003fe0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c00ba70 con 0x7ff02c004d40 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff02c00bc40 con 0x7ff0480cd650 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.545+0000 7ff047400640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff02c003fe0 0x7ff02c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047e00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff02c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047400640 1 -- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 msgr2=0x7ff02c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047400640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0480cd650 0x7ff02c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047400640 1 -- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c004d40 msgr2=0x7ff02c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047400640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c004d40 0x7ff02c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.546+0000 7ff047400640 1 -- 172.21.15.88:0/2531640231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c003520 con 0x7ff02c003fe0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.547+0000 7ff047400640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff02c003fe0 0x7ff02c004800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff03400ba30 tx=0x7ff02c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26473317ffbab780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.547+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034030070 con 0x7ff02c003fe0 2024-04-26T20:00:46.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.547+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03402ade0 con 0x7ff02c003fe0 2024-04-26T20:00:46.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.547+0000 7ff046a00640 1 -- 172.21.15.88:0/2531640231 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03400b600 con 0x7ff02c003fe0 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 -- 172.21.15.88:0/2531640231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff02c003fe0 msgr2=0x7ff02c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 --2- 172.21.15.88:0/2531640231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff02c003fe0 0x7ff02c004800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff03400ba30 tx=0x7ff02c005830 comp rx=0 tx=0).stop 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ce00640 1 -- 172.21.15.88:0/2531640231 reap_dead start 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 -- 172.21.15.88:0/2531640231 shutdown_connections 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 -- 172.21.15.88:0/2531640231 >> 172.21.15.88:0/2531640231 conn(0x7ff0480b2d50 msgr2=0x7ff0480b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 -- 172.21.15.88:0/2531640231 shutdown_connections 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.548+0000 7ff04ee00640 1 -- 172.21.15.88:0/2531640231 wait complete. 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.550+0000 7ff04ee00640 1 Processor -- start 2024-04-26T20:00:46.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.550+0000 7ff04ee00640 1 -- start start 2024-04-26T20:00:46.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 0x7ff04814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff048155b20 0x7ff04815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff04815d1d0 con 0x7ff04814ea40 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0480b0240 con 0x7ff02c003fe0 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff04815d350 con 0x7ff048155b20 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47482/0 (socket says 172.21.15.88:47482) 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 -- 172.21.15.88:0/1965919717 learned_addr learned my addr 172.21.15.88:0/1965919717 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff047e00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 0x7ff04814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 -- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff048155b20 msgr2=0x7ff04815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff047400640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff048155b20 0x7ff04815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff048155b20 0x7ff04815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 -- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 msgr2=0x7ff04814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 0x7ff04814b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff04ce00640 1 -- 172.21.15.88:0/1965919717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04809c540 con 0x7ff04814ea40 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.551+0000 7ff047e00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 0x7ff04814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:46.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff04ce00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0480b3900 tx=0x7ff04000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040007110 con 0x7ff04814ea40 2024-04-26T20:00:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0400072b0 con 0x7ff04814ea40 2024-04-26T20:00:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04809c820 con 0x7ff04814ea40 2024-04-26T20:00:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400076a0 con 0x7ff04814ea40 2024-04-26T20:00:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.552+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04814bb10 con 0x7ff04814ea40 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.553+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04809c360 con 0x7ff04814ea40 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.554+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff040011070 con 0x7ff04814ea40 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.555+0000 7ff03fe00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 0x7ff02806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.555+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7ff04008b3a0 con 0x7ff04814ea40 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.555+0000 7ff047e00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 0x7ff02806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.556+0000 7ff047e00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 0x7ff02806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff038006fd0 tx=0x7ff038008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.559+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff04809c360 con 0x7ff04814ea40 2024-04-26T20:00:46.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.761+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff048069540 con 0x7ff04814ea40 2024-04-26T20:00:46.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.763+0000 7ff03fe00640 1 -- 172.21.15.88:0/1965919717 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7ff040019060 con 0x7ff04814ea40 2024-04-26T20:00:46.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:46.765 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:46.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 msgr2=0x7ff02806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 0x7ff02806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff038006fd0 tx=0x7ff038008040 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 msgr2=0x7ff04815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff0480b3900 tx=0x7ff04000cf00 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 shutdown_connections 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04814ea40 0x7ff04815b500 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff02806c8f0 0x7ff02806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff048155b20 0x7ff04815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 --2- 172.21.15.88:0/1965919717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff02c003fe0 0x7ff04814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 >> 172.21.15.88:0/1965919717 conn(0x7ff0480b2d50 msgr2=0x7ff0480b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 shutdown_connections 2024-04-26T20:00:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.766+0000 7ff04ee00640 1 -- 172.21.15.88:0/1965919717 wait complete. 2024-04-26T20:00:46.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:46.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:46.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.989+0000 7fd5f9600640 1 Processor -- start 2024-04-26T20:00:46.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.989+0000 7fd5f9600640 1 -- start start 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f415d100 0x7fd5f406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f9600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd5f40add80 0x7fd5f406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f9600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5f40af800 con 0x7fd5f40b5ca0 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f9600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd5f40ae340 con 0x7fd5f40add80 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f40afaa0 con 0x7fd5f415d100 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5f3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd5f40add80 0x7fd5f406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5ebe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd5f40b5ca0 0x7fd5f4072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48004/0 (socket says 172.21.15.88:48004) 2024-04-26T20:00:46.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.991+0000 7fd5ebe00640 1 -- 172.21.15.88:0/2249512504 learned_addr learned my addr 172.21.15.88:0/2249512504 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f2a00640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f415d100 0x7fd5f406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f3400640 1 -- 172.21.15.88:0/2249512504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f415d100 msgr2=0x7fd5f406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f3400640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f415d100 0x7fd5f406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f3400640 1 -- 172.21.15.88:0/2249512504 >> v1:172.21.15.88:6789/0 conn(0x7fd5f40b5ca0 legacy=0x7fd5f4072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f3400640 1 -- 172.21.15.88:0/2249512504 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f40afd80 con 0x7fd5f40add80 2024-04-26T20:00:46.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.992+0000 7fd5f2000640 1 -- 172.21.15.88:0/2249512504 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085206046 0 0) 0x7fd5f40af800 con 0x7fd5f40b5ca0 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.993+0000 7fd5f3400640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40add80 0x7fd5f406ccb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009d90 tx=0x7fd5ec02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c8e19aab86f5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.993+0000 7fd5f2000640 1 -- 172.21.15.88:0/2249512504 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec031600 con 0x7fd5f40add80 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.993+0000 7fd5f2000640 1 -- 172.21.15.88:0/2249512504 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ec0317a0 con 0x7fd5f40add80 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.993+0000 7fd5f2000640 1 -- 172.21.15.88:0/2249512504 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec031b70 con 0x7fd5f40add80 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 -- 172.21.15.88:0/2249512504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40add80 msgr2=0x7fd5f406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40add80 0x7fd5f406ccb0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009d90 tx=0x7fd5ec02a5d0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 -- 172.21.15.88:0/2249512504 shutdown_connections 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40add80 0x7fd5f406ccb0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 --2- 172.21.15.88:0/2249512504 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f415d100 0x7fd5f406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 -- 172.21.15.88:0/2249512504 >> 172.21.15.88:0/2249512504 conn(0x7fd5f40b2d50 msgr2=0x7fd5f40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 -- 172.21.15.88:0/2249512504 shutdown_connections 2024-04-26T20:00:46.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.994+0000 7fd5f9600640 1 -- 172.21.15.88:0/2249512504 wait complete. 2024-04-26T20:00:46.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.996+0000 7fd5f9600640 1 Processor -- start 2024-04-26T20:00:46.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.996+0000 7fd5f9600640 1 -- start start 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f40add80 0x7fd5f409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 0x7fd5f414d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f409ee50 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40b0340 con 0x7fd5f415d100 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f9600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd5f4160b70 con 0x7fd5f40add80 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47488/0 (socket says 172.21.15.88:47488) 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5ebe00640 1 -- 172.21.15.88:0/1788948464 learned_addr learned my addr 172.21.15.88:0/1788948464 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.997+0000 7fd5f2a00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f40add80 0x7fd5f409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5f3400640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 0x7fd5f414d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 -- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f40add80 msgr2=0x7fd5f409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f40add80 0x7fd5f409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 -- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 msgr2=0x7fd5f414d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 0x7fd5f414d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 -- 172.21.15.88:0/1788948464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f415dfa0 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5f3400640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 0x7fd5f414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5ebe00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c430 tx=0x7fd5dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0074e0 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5dc0047e0 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f415e280 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004bd0 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.998+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f415ed30 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:46.999+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f4069650 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.000+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5dc011070 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.001+0000 7fd5eaa00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 0x7fd5d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.001+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fd5dc08b960 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.002+0000 7fd5f2a00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 0x7fd5d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.002+0000 7fd5f2a00640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 0x7fd5d006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.006+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd5dc059c60 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.203+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5f4073d00 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.204+0000 7fd5eaa00640 1 -- 172.21.15.88:0/1788948464 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fd5dc059560 con 0x7fd5f40b5ca0 2024-04-26T20:00:47.206 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 msgr2=0x7fd5d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 0x7fd5d006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4006fd0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 msgr2=0x7fd5f415d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c430 tx=0x7fd5dc00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 shutdown_connections 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5f415d100 0x7fd5f414d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5f40b5ca0 0x7fd5f415d4e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd5d006c8f0 0x7fd5d006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 --2- 172.21.15.88:0/1788948464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5f40add80 0x7fd5f409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 >> 172.21.15.88:0/1788948464 conn(0x7fd5f40b2d50 msgr2=0x7fd5f40b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.207+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 shutdown_connections 2024-04-26T20:00:47.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.208+0000 7fd5f9600640 1 -- 172.21.15.88:0/1788948464 wait complete. 2024-04-26T20:00:47.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.421+0000 7f9632a00640 1 Processor -- start 2024-04-26T20:00:47.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.421+0000 7f9632a00640 1 -- start start 2024-04-26T20:00:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.422+0000 7f9632a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f962c161140 0x7f962c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.422+0000 7f9632a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 0x7f962c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.422+0000 7f9632a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f962c0b4d20 con 0x7f962c160d60 2024-04-26T20:00:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.422+0000 7f9632a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f962c146c30 con 0x7f962c161140 2024-04-26T20:00:47.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.422+0000 7f9632a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f962c0b5000 con 0x7f962c146280 2024-04-26T20:00:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f962c161140 0x7f962c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f962c161140 0x7f962c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47492/0 (socket says 172.21.15.88:47492) 2024-04-26T20:00:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 -- 172.21.15.88:0/1839010233 learned_addr learned my addr 172.21.15.88:0/1839010233 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9632000640 1 --2- 172.21.15.88:0/1839010233 >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 0x7f962c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 -- 172.21.15.88:0/1839010233 >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 msgr2=0x7f962c1466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 --2- 172.21.15.88:0/1839010233 >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 0x7f962c1466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 -- 172.21.15.88:0/1839010233 >> v1:172.21.15.132:6789/0 conn(0x7f962c160d60 legacy=0x7f962c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9630c00640 1 -- 172.21.15.88:0/1839010233 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c0b52e0 con 0x7f962c161140 2024-04-26T20:00:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.423+0000 7f9632000640 1 --2- 172.21.15.88:0/1839010233 >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 0x7f962c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.424+0000 7f9630c00640 1 --2- 172.21.15.88:0/1839010233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c161140 0x7f962c06dee0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9618009870 tx=0x7f961802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c37add1460ae211 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.425+0000 7f962be00640 1 -- 172.21.15.88:0/1839010233 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618037070 con 0x7f962c161140 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.425+0000 7f962be00640 1 -- 172.21.15.88:0/1839010233 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9618032440 con 0x7f962c161140 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.425+0000 7f962be00640 1 -- 172.21.15.88:0/1839010233 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9618032810 con 0x7f962c161140 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.425+0000 7f9632a00640 1 -- 172.21.15.88:0/1839010233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c161140 msgr2=0x7f962c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.425+0000 7f9632a00640 1 --2- 172.21.15.88:0/1839010233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c161140 0x7f962c06dee0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9618009870 tx=0x7f961802a250 comp rx=0 tx=0).stop 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 -- 172.21.15.88:0/1839010233 shutdown_connections 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 --2- 172.21.15.88:0/1839010233 >> v2:172.21.15.132:3300/0 conn(0x7f962c146280 0x7f962c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 --2- 172.21.15.88:0/1839010233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c161140 0x7f962c06dee0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 -- 172.21.15.88:0/1839010233 >> 172.21.15.88:0/1839010233 conn(0x7f962c0b2900 msgr2=0x7f962c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 -- 172.21.15.88:0/1839010233 shutdown_connections 2024-04-26T20:00:47.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.426+0000 7f9632a00640 1 -- 172.21.15.88:0/1839010233 wait complete. 2024-04-26T20:00:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.428+0000 7f9632a00640 1 Processor -- start 2024-04-26T20:00:47.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.428+0000 7f9632a00640 1 -- start start 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962c160d60 0x7f962c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962c161140 0x7f962c13b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f962c06e420 con 0x7f962c146280 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0cf6f0 con 0x7f962c161140 2024-04-26T20:00:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9632a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f962c06e5a0 con 0x7f962c160d60 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9631600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9631600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47500/0 (socket says 172.21.15.88:47500) 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.429+0000 7f9631600640 1 -- 172.21.15.88:0/101905878 learned_addr learned my addr 172.21.15.88:0/101905878 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9630c00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962c160d60 0x7f962c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 -- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962c160d60 msgr2=0x7f962c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962c160d60 0x7f962c159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 -- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962c161140 msgr2=0x7f962c13b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962c161140 0x7f962c13b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 -- 172.21.15.88:0/101905878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1f3b60 con 0x7f962c146280 2024-04-26T20:00:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9631600640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f962000c7f0 tx=0x7f962000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620016020 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962000aa50 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962000ae40 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.430+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c140420 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.431+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1409e0 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.432+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0693c0 con 0x7f962c146280 2024-04-26T20:00:47.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.433+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9620019020 con 0x7f962c146280 2024-04-26T20:00:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.434+0000 7f9629600640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 0x7f960c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.434+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f962008ba10 con 0x7f962c146280 2024-04-26T20:00:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.434+0000 7f9630c00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 0x7f960c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.435+0000 7f9630c00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 0x7f960c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9618034000 tx=0x7f961802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.439+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9620020070 con 0x7f962c146280 2024-04-26T20:00:47.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.646+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f962c146a10 con 0x7f962c146280 2024-04-26T20:00:47.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.647+0000 7f9629600640 1 -- 172.21.15.88:0/101905878 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f9620059d10 con 0x7f962c146280 2024-04-26T20:00:47.649 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 msgr2=0x7f960c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 0x7f960c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9618034000 tx=0x7f961802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 msgr2=0x7f962c1f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f962000c7f0 tx=0x7f962000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 shutdown_connections 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f962c161140 0x7f962c13b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f960c06c8f0 0x7f960c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f962c160d60 0x7f962c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 --2- 172.21.15.88:0/101905878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f962c146280 0x7f962c1f3620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.649+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 >> 172.21.15.88:0/101905878 conn(0x7f962c0b2900 msgr2=0x7f962c145780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.650+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 shutdown_connections 2024-04-26T20:00:47.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.650+0000 7f9632a00640 1 -- 172.21.15.88:0/101905878 wait complete. 2024-04-26T20:00:47.668 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:47.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.867+0000 7f41bf800640 1 Processor -- start 2024-04-26T20:00:47.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.867+0000 7f41bf800640 1 -- start start 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41bf800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41b815d100 0x7f41b806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41bf800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b8072e30 con 0x7f41b80b5ca0 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41bf800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b80af800 con 0x7f41b80add80 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41bf800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f41b80ae340 con 0x7f41b815d100 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41afe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f41b80b5ca0 0x7f41b8072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48016/0 (socket says 172.21.15.88:48016) 2024-04-26T20:00:47.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.869+0000 7f41afe00640 1 -- 172.21.15.88:0/1675018022 learned_addr learned my addr 172.21.15.88:0/1675018022 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:47.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153374322 0 0) 0x7f41b8072e30 con 0x7f41b80b5ca0 2024-04-26T20:00:47.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f419c003520 con 0x7f41b80b5ca0 2024-04-26T20:00:47.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41bce00640 1 --2- 172.21.15.88:0/1675018022 >> v2:172.21.15.132:3300/0 conn(0x7f41b815d100 0x7f41b806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4247335121 0 0) 0x7f419c003520 con 0x7f41b80b5ca0 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 >> v2:172.21.15.132:3300/0 conn(0x7f41b815d100 msgr2=0x7f41b806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 --2- 172.21.15.88:0/1675018022 >> v2:172.21.15.132:3300/0 conn(0x7f41b815d100 0x7f41b806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 >> v1:172.21.15.132:6789/0 conn(0x7f41b80add80 legacy=0x7f41b8072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b80afaa0 con 0x7f41b80b5ca0 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f41a0001670 con 0x7f41b80b5ca0 2024-04-26T20:00:47.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 >> v1:172.21.15.88:6789/0 conn(0x7f41b80b5ca0 legacy=0x7f41b8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 0x7f419c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c004800 0x7f419c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f419c005210 0x7f419c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41b80ae340 con 0x7f419c003a00 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f419c00c160 con 0x7f419c005210 2024-04-26T20:00:47.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.870+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f419c00c360 con 0x7f419c004800 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 0x7f419c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41afe00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c004800 0x7f419c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bd800640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f419c005210 0x7f419c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 -- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c004800 msgr2=0x7f419c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c004800 0x7f419c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 -- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f419c005210 msgr2=0x7f419c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f419c005210 0x7f419c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bce00640 1 -- 172.21.15.88:0/1675018022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419c003520 con 0x7f419c003a00 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bd800640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f419c005210 0x7f419c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.871+0000 7f41bd800640 1 -- 172.21.15.88:0/1675018022 reap_dead start 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.872+0000 7f41bce00640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 0x7f419c003e20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ba30 tx=0x7f419c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f876dcc6a57d1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.872+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8030070 con 0x7f419c003a00 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.872+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a802ade0 con 0x7f419c003a00 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.872+0000 7f41b7e00640 1 -- 172.21.15.88:0/1675018022 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a800b600 con 0x7f419c003a00 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 -- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 msgr2=0x7f419c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 0x7f419c003e20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ba30 tx=0x7f419c005830 comp rx=0 tx=0).stop 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 -- 172.21.15.88:0/1675018022 shutdown_connections 2024-04-26T20:00:47.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 --2- 172.21.15.88:0/1675018022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f419c003a00 0x7f419c003e20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 -- 172.21.15.88:0/1675018022 >> 172.21.15.88:0/1675018022 conn(0x7f41b80b2d50 msgr2=0x7f41b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:47.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 -- 172.21.15.88:0/1675018022 shutdown_connections 2024-04-26T20:00:47.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.873+0000 7f41bf800640 1 -- 172.21.15.88:0/1675018022 wait complete. 2024-04-26T20:00:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.875+0000 7f41bf800640 1 Processor -- start 2024-04-26T20:00:47.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.876+0000 7f41bf800640 1 -- start start 2024-04-26T20:00:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c003a00 0x7f41b815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 0x7f41b81496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41b80b0340 con 0x7f41b809da60 2024-04-26T20:00:47.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f41b80b0060 con 0x7f41b81590e0 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bf800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f41b809ee50 con 0x7f419c003a00 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47526/0 (socket says 172.21.15.88:47526) 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 -- 172.21.15.88:0/3569472853 learned_addr learned my addr 172.21.15.88:0/3569472853 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bce00640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c003a00 0x7f41b815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41afe00640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 0x7f41b81496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 -- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c003a00 msgr2=0x7f41b815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c003a00 0x7f41b815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 -- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 msgr2=0x7f41b81496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 0x7f41b81496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41bd800640 1 -- 172.21.15.88:0/3569472853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b8159650 con 0x7f41b809da60 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.877+0000 7f41afe00640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 0x7f41b81496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:47.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41bd800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41b80d1dc0 tx=0x7f41b000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0018070 con 0x7f41b809da60 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b0003da0 con 0x7f41b809da60 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b8159930 con 0x7f41b809da60 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b00052b0 con 0x7f41b809da60 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.878+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b8159ef0 con 0x7f41b809da60 2024-04-26T20:00:47.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.879+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b8069650 con 0x7f41b809da60 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.880+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41b0005450 con 0x7f41b809da60 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.881+0000 7f41b5600640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 0x7f419406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.881+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f41b008b230 con 0x7f41b809da60 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.882+0000 7f41bce00640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 0x7f419406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.882+0000 7f41bce00640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 0x7f419406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ba00 tx=0x7f41a8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:47.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:47.885+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41b00595c0 con 0x7f41b809da60 2024-04-26T20:00:48.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.079+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41b8073d00 con 0x7f41b809da60 2024-04-26T20:00:48.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.080+0000 7f41b5600640 1 -- 172.21.15.88:0/3569472853 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f41b0058ec0 con 0x7f41b809da60 2024-04-26T20:00:48.082 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:48.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.082+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 msgr2=0x7f419406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.082+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 0x7f419406ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ba00 tx=0x7f41a8030000 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.082+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 msgr2=0x7f41b809df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.082+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41b80d1dc0 tx=0x7f41b000d9d0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 shutdown_connections 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41b809da60 0x7f41b809df10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41b81590e0 0x7f41b81496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f419406c920 0x7f419406ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 --2- 172.21.15.88:0/3569472853 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f419c003a00 0x7f41b815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 >> 172.21.15.88:0/3569472853 conn(0x7f41b80b2d50 msgr2=0x7f41b80b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 shutdown_connections 2024-04-26T20:00:48.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.083+0000 7f41bf800640 1 -- 172.21.15.88:0/3569472853 wait complete. 2024-04-26T20:00:48.102 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:00:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.295+0000 7fd8f3c00640 1 Processor -- start 2024-04-26T20:00:48.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.296+0000 7fd8f3c00640 1 -- start start 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f3c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8ec15f420 0x7fd8ec0d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 0x7fd8ec0d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f3c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ec15f800 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f3c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ec0d2c70 con 0x7fd8ec15f420 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d3480 con 0x7fd8ec0d3680 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f1c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd8ec0cfb90 0x7fd8ec0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:48032/0 (socket says 172.21.15.88:48032) 2024-04-26T20:00:48.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.297+0000 7fd8f1c00640 1 -- 172.21.15.88:0/2483741386 learned_addr learned my addr 172.21.15.88:0/2483741386 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8f1200640 1 --2- 172.21.15.88:0/2483741386 >> v2:172.21.15.132:3300/0 conn(0x7fd8ec15f420 0x7fd8ec0d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8ebe00640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 0x7fd8ec0d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3843232599 0 0) 0x7fd8ec15f800 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8cc003520 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4130390331 0 0) 0x7fd8cc003520 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 msgr2=0x7fd8ec0d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 0x7fd8ec0d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 >> v2:172.21.15.132:3300/0 conn(0x7fd8ec15f420 msgr2=0x7fd8ec0d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 --2- 172.21.15.88:0/2483741386 >> v2:172.21.15.132:3300/0 conn(0x7fd8ec15f420 0x7fd8ec0d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec0d4020 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd8e4001670 con 0x7fd8ec0cfb90 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 >> v1:172.21.15.88:6789/0 conn(0x7fd8ec0cfb90 legacy=0x7fd8ec0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.298+0000 7fd8eb400640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8cc0039b0 0x7fd8cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8eb400640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc004800 0x7fd8cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8eb400640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 0x7fd8cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d2c70 con 0x7fd8cc0039b0 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d3480 con 0x7fd8cc004800 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc00b9c0 con 0x7fd8ec0d3680 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8cc0039b0 0x7fd8cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 -- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 msgr2=0x7fd8cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:48.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0d3680 0x7fd8cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 -- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc004800 msgr2=0x7fd8cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:48.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc004800 0x7fd8cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.299+0000 7fd8f1200640 1 -- 172.21.15.88:0/2483741386 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc003520 con 0x7fd8cc0039b0 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.300+0000 7fd8f1200640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8cc0039b0 0x7fd8cc003dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009af0 tx=0x7fd8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a53686cea8dabe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.300+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc036070 con 0x7fd8cc0039b0 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.300+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc031400 con 0x7fd8cc0039b0 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.300+0000 7fd8eb400640 1 -- 172.21.15.88:0/2483741386 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc009690 con 0x7fd8cc0039b0 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 -- 172.21.15.88:0/2483741386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8cc0039b0 msgr2=0x7fd8cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/2483741386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8cc0039b0 0x7fd8cc003dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009af0 tx=0x7fd8cc005830 comp rx=0 tx=0).stop 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f1c00640 1 -- 172.21.15.88:0/2483741386 reap_dead start 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 -- 172.21.15.88:0/2483741386 shutdown_connections 2024-04-26T20:00:48.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 -- 172.21.15.88:0/2483741386 >> 172.21.15.88:0/2483741386 conn(0x7fd8ec09e3e0 msgr2=0x7fd8ec0d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:48.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 -- 172.21.15.88:0/2483741386 shutdown_connections 2024-04-26T20:00:48.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.301+0000 7fd8f3c00640 1 -- 172.21.15.88:0/2483741386 wait complete. 2024-04-26T20:00:48.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.303+0000 7fd8f3c00640 1 Processor -- start 2024-04-26T20:00:48.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.303+0000 7fd8f3c00640 1 -- start start 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc0039b0 0x7fd8ec0b9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0cd760 0x7fd8ec160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0c8d20 con 0x7fd8ec154150 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec0d31a0 con 0x7fd8cc0039b0 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f3c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec0c9000 con 0x7fd8ec0cd760 2024-04-26T20:00:48.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47532/0 (socket says 172.21.15.88:47532) 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.304+0000 7fd8f1c00640 1 -- 172.21.15.88:0/1447368917 learned_addr learned my addr 172.21.15.88:0/1447368917 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8ebe00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0cd760 0x7fd8ec160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1200640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc0039b0 0x7fd8ec0b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 -- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0cd760 msgr2=0x7fd8ec160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0cd760 0x7fd8ec160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 -- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc0039b0 msgr2=0x7fd8ec0b9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc0039b0 0x7fd8ec0b9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 -- 172.21.15.88:0/1447368917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec154b20 con 0x7fd8ec154150 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f1c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4001050 tx=0x7fd8e400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e40072d0 con 0x7fd8ec154150 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e4007490 con 0x7fd8ec154150 2024-04-26T20:00:48.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4004a20 con 0x7fd8ec154150 2024-04-26T20:00:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.305+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec0b9b80 con 0x7fd8ec154150 2024-04-26T20:00:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.306+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec0ba140 con 0x7fd8ec154150 2024-04-26T20:00:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.307+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec069540 con 0x7fd8ec154150 2024-04-26T20:00:48.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.308+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8e4011070 con 0x7fd8ec154150 2024-04-26T20:00:48.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.308+0000 7fd8e8c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 0x7fd8c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.309+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fd8e408aff0 con 0x7fd8ec154150 2024-04-26T20:00:48.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.309+0000 7fd8f1200640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 0x7fd8c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.309+0000 7fd8f1200640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 0x7fd8c806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc033000 tx=0x7fd8dc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.313+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8e40593a0 con 0x7fd8ec154150 2024-04-26T20:00:48.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.525+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd8ec06d230 con 0x7fd8ec154150 2024-04-26T20:00:48.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.525+0000 7fd8e8c00640 1 -- 172.21.15.88:0/1447368917 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd8e4058ca0 con 0x7fd8ec154150 2024-04-26T20:00:48.527 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 msgr2=0x7fd8c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 0x7fd8c806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc033000 tx=0x7fd8dc008030 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 msgr2=0x7fd8ec1545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4001050 tx=0x7fd8e400cb90 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 shutdown_connections 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8ec154150 0x7fd8ec1545e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c806c8f0 0x7fd8c806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8ec0cd760 0x7fd8ec160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 --2- 172.21.15.88:0/1447368917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8cc0039b0 0x7fd8ec0b9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 >> 172.21.15.88:0/1447368917 conn(0x7fd8ec09e3e0 msgr2=0x7fd8ec0cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 shutdown_connections 2024-04-26T20:00:48.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.528+0000 7fd8f3c00640 1 -- 172.21.15.88:0/1447368917 wait complete. 2024-04-26T20:00:48.548 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi132.front.sepia.ceph.com... 2024-04-26T20:00:48.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:48.548 DEBUG:teuthology.orchestra.run.smithi132:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-26T20:00:48.644 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:48.645 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:00:48.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:00:48.645 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T20:00:48.699 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:00:48.699 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:48.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.910+0000 7f23bfe00640 1 Processor -- start 2024-04-26T20:00:48.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.910+0000 7f23bfe00640 1 -- start start 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.910+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00d0e70 0x7f23c00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.910+0000 7f23bfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f23c00b3b20 0x7f23c00b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bfe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c0146a60 con 0x7f23c0146680 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bfe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f23c006e970 con 0x7f23c00b3b20 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00b60a0 con 0x7f23c00d0e70 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f23c0146680 0x7f23c00b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36030/0 (socket says 172.21.15.88:36030) 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bea00640 1 -- 172.21.15.88:0/3084470802 learned_addr learned my addr 172.21.15.88:0/3084470802 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bf400640 1 --2- 172.21.15.88:0/3084470802 >> v2:172.21.15.88:3300/0 conn(0x7f23c00b3b20 0x7f23c00b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bf400640 1 -- 172.21.15.88:0/3084470802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00d0e70 msgr2=0x7f23c00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bf400640 1 --2- 172.21.15.88:0/3084470802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00d0e70 0x7f23c00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bf400640 1 -- 172.21.15.88:0/3084470802 >> v1:172.21.15.88:6789/0 conn(0x7f23c0146680 legacy=0x7f23c00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.911+0000 7f23bf400640 1 -- 172.21.15.88:0/3084470802 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c006dd60 con 0x7f23c00b3b20 2024-04-26T20:00:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.912+0000 7f23bf400640 1 --2- 172.21.15.88:0/3084470802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c00b3b20 0x7f23c00b3fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d90 tx=0x7f23b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2264687252792d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.912+0000 7f23bd600640 1 -- 172.21.15.88:0/3084470802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007740 con 0x7f23c00b3b20 2024-04-26T20:00:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bd600640 1 -- 172.21.15.88:0/3084470802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80078e0 con 0x7f23c00b3b20 2024-04-26T20:00:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bd600640 1 -- 172.21.15.88:0/3084470802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007cb0 con 0x7f23c00b3b20 2024-04-26T20:00:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 -- 172.21.15.88:0/3084470802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c00b3b20 msgr2=0x7f23c00b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 --2- 172.21.15.88:0/3084470802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c00b3b20 0x7f23c00b3fb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d90 tx=0x7f23b802a660 comp rx=0 tx=0).stop 2024-04-26T20:00:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 -- 172.21.15.88:0/3084470802 shutdown_connections 2024-04-26T20:00:48.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 --2- 172.21.15.88:0/3084470802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c00b3b20 0x7f23c00b3fb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 --2- 172.21.15.88:0/3084470802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00d0e70 0x7f23c00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 -- 172.21.15.88:0/3084470802 >> 172.21.15.88:0/3084470802 conn(0x7f23c006c740 msgr2=0x7f23c006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 -- 172.21.15.88:0/3084470802 shutdown_connections 2024-04-26T20:00:48.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.913+0000 7f23bfe00640 1 -- 172.21.15.88:0/3084470802 wait complete. 2024-04-26T20:00:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.915+0000 7f23bfe00640 1 Processor -- start 2024-04-26T20:00:48.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.916+0000 7f23bfe00640 1 -- start start 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 0x7f23c013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 0x7f23c0141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f23c006e320 con 0x7f23c0146680 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f23c00d17f0 con 0x7f23c00d0e70 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f23c006e600 con 0x7f23c00b3b20 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53750/0 (socket says 172.21.15.88:53750) 2024-04-26T20:00:48.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 -- 172.21.15.88:0/1840705489 learned_addr learned my addr 172.21.15.88:0/1840705489 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23be000640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 0x7f23c0141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bea00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 0x7f23c013ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 -- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 msgr2=0x7f23c013ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 0x7f23c013ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.917+0000 7f23bf400640 1 -- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 msgr2=0x7f23c0141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bf400640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 0x7f23c0141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bf400640 1 -- 172.21.15.88:0/1840705489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c013c580 con 0x7f23c0146680 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bea00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 0x7f23c013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23be000640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 0x7f23c0141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bf400640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009eb0 tx=0x7f23b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8040070 con 0x7f23c0146680 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b8031c90 con 0x7f23c0146680 2024-04-26T20:00:48.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8045680 con 0x7f23c0146680 2024-04-26T20:00:48.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c013c2a0 con 0x7f23c0146680 2024-04-26T20:00:48.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.918+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c0136330 con 0x7f23c0146680 2024-04-26T20:00:48.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.919+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c00693c0 con 0x7f23c0146680 2024-04-26T20:00:48.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.920+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23b8005d10 con 0x7f23c0146680 2024-04-26T20:00:48.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.921+0000 7f23b6a00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 0x7f23a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:48.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.921+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f23b80acf90 con 0x7f23c0146680 2024-04-26T20:00:48.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.921+0000 7f23bea00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 0x7f23a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:48.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.922+0000 7f23bea00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 0x7f23a006ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000f60 tx=0x7f23b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:48.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:48.927+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f23b804c020 con 0x7f23c0146680 2024-04-26T20:00:49.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.165+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23c015eff0 con 0x7f23c0146680 2024-04-26T20:00:49.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.165+0000 7f23b6a00640 1 -- 172.21.15.88:0/1840705489 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f23b8038350 con 0x7f23c0146680 2024-04-26T20:00:49.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:49.168 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:49.168 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 msgr2=0x7f23a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 0x7f23a006ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000f60 tx=0x7f23b0009040 comp rx=0 tx=0).stop 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 msgr2=0x7f23c013c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009eb0 tx=0x7f23b8004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 shutdown_connections 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23c0146680 0x7f23c013c7d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23c00d0e70 0x7f23c0141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f23a006c920 0x7f23a006ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 --2- 172.21.15.88:0/1840705489 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f23c00b3b20 0x7f23c013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 >> 172.21.15.88:0/1840705489 conn(0x7f23c006c740 msgr2=0x7f23c00cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 shutdown_connections 2024-04-26T20:00:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.169+0000 7f23bfe00640 1 -- 172.21.15.88:0/1840705489 wait complete. 2024-04-26T20:00:49.190 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:49.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.390+0000 7f3925200640 1 Processor -- start 2024-04-26T20:00:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.390+0000 7f3925200640 1 -- start start 2024-04-26T20:00:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.390+0000 7f3925200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.390+0000 7f3925200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39200b5660 0x7f39200ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f3925200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39200d1790 con 0x7f39200b5d40 2024-04-26T20:00:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f3925200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f39200727f0 con 0x7f39200b5660 2024-04-26T20:00:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f3925200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f39200b4a10 con 0x7f39200d0e70 2024-04-26T20:00:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f391f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f391f400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:36506/0 (socket says 172.21.15.88:36506) 2024-04-26T20:00:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f391f400640 1 -- 172.21.15.88:0/1948074350 learned_addr learned my addr 172.21.15.88:0/1948074350 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.391+0000 7f3917e00640 1 --2- 172.21.15.88:0/1948074350 >> v2:172.21.15.132:3300/0 conn(0x7f39200b5660 0x7f39200ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.392+0000 7f391f400640 1 -- 172.21.15.88:0/1948074350 >> v2:172.21.15.132:3300/0 conn(0x7f39200b5660 msgr2=0x7f39200ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.392+0000 7f391f400640 1 --2- 172.21.15.88:0/1948074350 >> v2:172.21.15.132:3300/0 conn(0x7f39200b5660 0x7f39200ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.392+0000 7f391f400640 1 -- 172.21.15.88:0/1948074350 >> v1:172.21.15.132:6789/0 conn(0x7f39200b5d40 legacy=0x7f39200aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.392+0000 7f391f400640 1 -- 172.21.15.88:0/1948074350 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39200b4cf0 con 0x7f39200d0e70 2024-04-26T20:00:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.392+0000 7f391ea00640 1 -- 172.21.15.88:0/1948074350 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3251314815 0 0) 0x7f39200d1790 con 0x7f39200b5d40 2024-04-26T20:00:49.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.394+0000 7f391f400640 1 --2- 172.21.15.88:0/1948074350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3910009870 tx=0x7f391002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee744c6af0bc9ea9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.394+0000 7f391ea00640 1 -- 172.21.15.88:0/1948074350 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910037070 con 0x7f39200d0e70 2024-04-26T20:00:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.394+0000 7f391ea00640 1 -- 172.21.15.88:0/1948074350 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910032440 con 0x7f39200d0e70 2024-04-26T20:00:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.394+0000 7f391ea00640 1 -- 172.21.15.88:0/1948074350 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910032810 con 0x7f39200d0e70 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 -- 172.21.15.88:0/1948074350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 msgr2=0x7f39200d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 --2- 172.21.15.88:0/1948074350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3910009870 tx=0x7f391002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 -- 172.21.15.88:0/1948074350 shutdown_connections 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 --2- 172.21.15.88:0/1948074350 >> v2:172.21.15.132:3300/0 conn(0x7f39200b5660 0x7f39200ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 --2- 172.21.15.88:0/1948074350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200d0e70 0x7f39200d1250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 -- 172.21.15.88:0/1948074350 >> 172.21.15.88:0/1948074350 conn(0x7f392006c740 msgr2=0x7f392006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 -- 172.21.15.88:0/1948074350 shutdown_connections 2024-04-26T20:00:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.395+0000 7f3925200640 1 -- 172.21.15.88:0/1948074350 wait complete. 2024-04-26T20:00:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.397+0000 7f3925200640 1 Processor -- start 2024-04-26T20:00:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.397+0000 7f3925200640 1 -- start start 2024-04-26T20:00:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39200b5660 0x7f39201495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200b5d40 0x7f3920156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39200b05b0 con 0x7f39200d0e70 2024-04-26T20:00:49.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f39200b0430 con 0x7f39200b5660 2024-04-26T20:00:49.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f3925200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f392015f1a0 con 0x7f39200b5d40 2024-04-26T20:00:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f391fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f391fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53756/0 (socket says 172.21.15.88:53756) 2024-04-26T20:00:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.398+0000 7f391fe00640 1 -- 172.21.15.88:0/816944190 learned_addr learned my addr 172.21.15.88:0/816944190 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391f400640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39200b5660 0x7f39201495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f3917e00640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200b5d40 0x7f3920156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 -- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200b5d40 msgr2=0x7f3920156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200b5d40 0x7f3920156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 -- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39200b5660 msgr2=0x7f39201495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39200b5660 0x7f39201495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 -- 172.21.15.88:0/816944190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3920152a00 con 0x7f39200d0e70 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f391fe00640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f391800c780 tx=0x7f391800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918003b70 con 0x7f39200d0e70 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3918003d10 con 0x7f39200d0e70 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918005270 con 0x7f39200d0e70 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.399+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3920157b10 con 0x7f39200d0e70 2024-04-26T20:00:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.400+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3920160c50 con 0x7f39200d0e70 2024-04-26T20:00:49.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.401+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200693c0 con 0x7f39200d0e70 2024-04-26T20:00:49.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.402+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3918005410 con 0x7f39200d0e70 2024-04-26T20:00:49.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.403+0000 7f3917400640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 0x7f38fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.403+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f391808bb80 con 0x7f39200d0e70 2024-04-26T20:00:49.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.403+0000 7f391f400640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 0x7f38fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.404+0000 7f391f400640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 0x7f38fc06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3910034000 tx=0x7f391002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.408+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3918059e80 con 0x7f39200d0e70 2024-04-26T20:00:49.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.607+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f392006cbc0 con 0x7f39200d0e70 2024-04-26T20:00:49.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.610+0000 7f3917400640 1 -- 172.21.15.88:0/816944190 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f3918059780 con 0x7f39200d0e70 2024-04-26T20:00:49.613 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.612+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 msgr2=0x7f38fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.612+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 0x7f38fc06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3910034000 tx=0x7f391002e000 comp rx=0 tx=0).stop 2024-04-26T20:00:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.612+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 msgr2=0x7f3920157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.612+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f391800c780 tx=0x7f391800cb80 comp rx=0 tx=0).stop 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 shutdown_connections 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39200d0e70 0x7f3920157490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38fc06c8f0 0x7f38fc06edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39200b5d40 0x7f3920156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 --2- 172.21.15.88:0/816944190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39200b5660 0x7f39201495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 >> 172.21.15.88:0/816944190 conn(0x7f392006c740 msgr2=0x7f39200b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 shutdown_connections 2024-04-26T20:00:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.613+0000 7f3925200640 1 -- 172.21.15.88:0/816944190 wait complete. 2024-04-26T20:00:49.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:59:44.098424+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:59:44.695841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:00:49.635 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:00:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.827+0000 7f26bfe00640 1 Processor -- start 2024-04-26T20:00:49.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.828+0000 7f26bfe00640 1 -- start start 2024-04-26T20:00:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bfe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26c015f7f0 0x7f26c00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26c00d2c50 0x7f26c00d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bfe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c00d0170 con 0x7f26c015fbd0 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bfe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f26c00d0450 con 0x7f26c00d2c50 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bfe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f26c00d0730 con 0x7f26c015f7f0 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bf400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26c00d2c50 0x7f26c00d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bf400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26c00d2c50 0x7f26c00d30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53758/0 (socket says 172.21.15.88:53758) 2024-04-26T20:00:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bf400640 1 -- 172.21.15.88:0/3183421491 learned_addr learned my addr 172.21.15.88:0/3183421491 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bf400640 1 -- 172.21.15.88:0/3183421491 >> v2:172.21.15.132:3300/0 conn(0x7f26c015f7f0 msgr2=0x7f26c00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.829+0000 7f26bf400640 1 --2- 172.21.15.88:0/3183421491 >> v2:172.21.15.132:3300/0 conn(0x7f26c015f7f0 0x7f26c00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.830+0000 7f26bf400640 1 -- 172.21.15.88:0/3183421491 >> v1:172.21.15.88:6789/0 conn(0x7f26c015fbd0 legacy=0x7f26c00d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.830+0000 7f26bf400640 1 -- 172.21.15.88:0/3183421491 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00d0a10 con 0x7f26c00d2c50 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26bf400640 1 --2- 172.21.15.88:0/3183421491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c00d2c50 0x7f26c00d30c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26c008dbd0 tx=0x7f26b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba736c2575b73fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26be000640 1 -- 172.21.15.88:0/3183421491 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8036070 con 0x7f26c00d2c50 2024-04-26T20:00:49.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26be000640 1 -- 172.21.15.88:0/3183421491 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8031600 con 0x7f26c00d2c50 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26be000640 1 -- 172.21.15.88:0/3183421491 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80319d0 con 0x7f26c00d2c50 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26bfe00640 1 -- 172.21.15.88:0/3183421491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c00d2c50 msgr2=0x7f26c00d30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.831+0000 7f26bfe00640 1 --2- 172.21.15.88:0/3183421491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c00d2c50 0x7f26c00d30c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26c008dbd0 tx=0x7f26b802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 -- 172.21.15.88:0/3183421491 shutdown_connections 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 --2- 172.21.15.88:0/3183421491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c00d2c50 0x7f26c00d30c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 --2- 172.21.15.88:0/3183421491 >> v2:172.21.15.132:3300/0 conn(0x7f26c015f7f0 0x7f26c00cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 -- 172.21.15.88:0/3183421491 >> 172.21.15.88:0/3183421491 conn(0x7f26c00d4f80 msgr2=0x7f26c00d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 -- 172.21.15.88:0/3183421491 shutdown_connections 2024-04-26T20:00:49.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.832+0000 7f26bfe00640 1 -- 172.21.15.88:0/3183421491 wait complete. 2024-04-26T20:00:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 Processor -- start 2024-04-26T20:00:49.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 -- start start 2024-04-26T20:00:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26c00d2c50 0x7f26c00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c015f7f0 0x7f26c0158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.834+0000 7f26bfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f26c015f430 con 0x7f26c015fbd0 2024-04-26T20:00:49.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bfe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0160050 con 0x7f26c015f7f0 2024-04-26T20:00:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bfe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f26c015f5b0 con 0x7f26c00d2c50 2024-04-26T20:00:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53762/0 (socket says 172.21.15.88:53762) 2024-04-26T20:00:49.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 -- 172.21.15.88:0/1346619500 learned_addr learned my addr 172.21.15.88:0/1346619500 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 -- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26c00d2c50 msgr2=0x7f26c00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26c00d2c50 0x7f26c00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 -- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c015f7f0 msgr2=0x7f26c0158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c015f7f0 0x7f26c0158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 -- 172.21.15.88:0/1346619500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c00b8e40 con 0x7f26c015fbd0 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bf400640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009f40 tx=0x7f26b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8036050 con 0x7f26c015fbd0 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b8042070 con 0x7f26c015fbd0 2024-04-26T20:00:49.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8009510 con 0x7f26c015fbd0 2024-04-26T20:00:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c00b82c0 con 0x7f26c015fbd0 2024-04-26T20:00:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.836+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c00b8880 con 0x7f26c015fbd0 2024-04-26T20:00:49.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.837+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c00693c0 con 0x7f26c015fbd0 2024-04-26T20:00:49.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.838+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26b8007bd0 con 0x7f26c015fbd0 2024-04-26T20:00:49.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.839+0000 7f26b6a00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 0x7f269c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:49.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.839+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f26b80acbd0 con 0x7f26c015fbd0 2024-04-26T20:00:49.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.840+0000 7f26bea00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 0x7f269c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:49.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.840+0000 7f26bea00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 0x7f269c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009670 tx=0x7f26b0006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:49.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:49.844+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26b807af60 con 0x7f26c015fbd0 2024-04-26T20:00:50.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.046+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26c008ea90 con 0x7f26c015fbd0 2024-04-26T20:00:50.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.047+0000 7f26b6a00640 1 -- 172.21.15.88:0/1346619500 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f26b807a860 con 0x7f26c015fbd0 2024-04-26T20:00:50.048 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 msgr2=0x7f269c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 0x7f269c06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009670 tx=0x7f26b0006bf0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 msgr2=0x7f26c00ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009f40 tx=0x7f26b8004a40 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 shutdown_connections 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26c015fbd0 0x7f26c00ce220 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c015f7f0 0x7f26c0158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f269c06c8f0 0x7f269c06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 --2- 172.21.15.88:0/1346619500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26c00d2c50 0x7f26c00cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 >> 172.21.15.88:0/1346619500 conn(0x7f26c00d4f80 msgr2=0x7f26c0160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:50.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 shutdown_connections 2024-04-26T20:00:50.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.050+0000 7f26bfe00640 1 -- 172.21.15.88:0/1346619500 wait complete. 2024-04-26T20:00:50.071 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:00:50.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.276+0000 7ff7fb400640 1 Processor -- start 2024-04-26T20:00:50.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.277+0000 7ff7fb400640 1 -- start start 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f415d100 0x7ff7f406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff7f40add80 0x7ff7f4072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f40af800 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff7f40ae340 con 0x7ff7f40add80 2024-04-26T20:00:50.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.278+0000 7ff7fb400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f40afaa0 con 0x7ff7f415d100 2024-04-26T20:00:50.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53776/0 (socket says 172.21.15.88:53776) 2024-04-26T20:00:50.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 -- 172.21.15.88:0/898910869 learned_addr learned my addr 172.21.15.88:0/898910869 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:50.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f8a00640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f415d100 0x7ff7f406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f9400640 1 --2- 172.21.15.88:0/898910869 >> v2:172.21.15.132:3300/0 conn(0x7ff7f40add80 0x7ff7f4072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 -- 172.21.15.88:0/898910869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f415d100 msgr2=0x7ff7f406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f415d100 0x7ff7f406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 -- 172.21.15.88:0/898910869 >> v2:172.21.15.132:3300/0 conn(0x7ff7f40add80 msgr2=0x7ff7f4072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 --2- 172.21.15.88:0/898910869 >> v2:172.21.15.132:3300/0 conn(0x7ff7f40add80 0x7ff7f4072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.279+0000 7ff7f3e00640 1 -- 172.21.15.88:0/898910869 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f40afd80 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7f3e00640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4073c70 tx=0x7ff7e002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f5f8d70f704739 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7f3400640 1 -- 172.21.15.88:0/898910869 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0037070 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7f3400640 1 -- 172.21.15.88:0/898910869 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e0032440 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7f3400640 1 -- 172.21.15.88:0/898910869 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0032850 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7fb400640 1 -- 172.21.15.88:0/898910869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f40b5ca0 msgr2=0x7ff7f406ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.281+0000 7ff7fb400640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4073c70 tx=0x7ff7e002a160 comp rx=0 tx=0).stop 2024-04-26T20:00:50.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 -- 172.21.15.88:0/898910869 shutdown_connections 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 --2- 172.21.15.88:0/898910869 >> v2:172.21.15.132:3300/0 conn(0x7ff7f40add80 0x7ff7f4072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f40b5ca0 0x7ff7f406ccb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 --2- 172.21.15.88:0/898910869 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f415d100 0x7ff7f406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 -- 172.21.15.88:0/898910869 >> 172.21.15.88:0/898910869 conn(0x7ff7f40b2d50 msgr2=0x7ff7f40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 -- 172.21.15.88:0/898910869 shutdown_connections 2024-04-26T20:00:50.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.282+0000 7ff7fb400640 1 -- 172.21.15.88:0/898910869 wait complete. 2024-04-26T20:00:50.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.283+0000 7ff7fb400640 1 Processor -- start 2024-04-26T20:00:50.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.284+0000 7ff7fb400640 1 -- start start 2024-04-26T20:00:50.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.284+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 0x7ff7f4158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.284+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f40b5ca0 0x7ff7f414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.284+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7fb400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f409ee50 con 0x7ff7f415d100 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7fb400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f40b0340 con 0x7ff7f40add80 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7fb400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f4160b70 con 0x7ff7f40b5ca0 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53782/0 (socket says 172.21.15.88:53782) 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 -- 172.21.15.88:0/610211048 learned_addr learned my addr 172.21.15.88:0/610211048 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:50.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f3e00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f40b5ca0 0x7ff7f414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f8a00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 0x7ff7f4158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 -- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f40b5ca0 msgr2=0x7ff7f414d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f40b5ca0 0x7ff7f414d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 -- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 msgr2=0x7ff7f4158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 0x7ff7f4158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f9400640 1 -- 172.21.15.88:0/610211048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f414dd70 con 0x7ff7f415d100 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.285+0000 7ff7f8a00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 0x7ff7f4158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7f9400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007e40 tx=0x7ff7ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014620 con 0x7ff7f415d100 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ec019070 con 0x7ff7f415d100 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014a20 con 0x7ff7f415d100 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f409d010 con 0x7ff7f415d100 2024-04-26T20:00:50.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.286+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f414fb30 con 0x7ff7f415d100 2024-04-26T20:00:50.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.287+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4069650 con 0x7ff7f415d100 2024-04-26T20:00:50.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.289+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7ec014bc0 con 0x7ff7f415d100 2024-04-26T20:00:50.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.290+0000 7ff7f0c00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.290+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7ff7ec08c590 con 0x7ff7f415d100 2024-04-26T20:00:50.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.290+0000 7ff7f8a00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.291+0000 7ff7f8a00640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 0x7ff7d406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006fd0 tx=0x7ff7e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.297+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7ec05a890 con 0x7ff7f415d100 2024-04-26T20:00:50.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.499+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff7f41581f0 con 0x7ff7f415d100 2024-04-26T20:00:50.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.501+0000 7ff7f0c00640 1 -- 172.21.15.88:0/610211048 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff7ec05a190 con 0x7ff7f415d100 2024-04-26T20:00:50.502 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:00:50.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 msgr2=0x7ff7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 0x7ff7d406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006fd0 tx=0x7ff7e8008040 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 msgr2=0x7ff7f4151630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007e40 tx=0x7ff7ec00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 shutdown_connections 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f415d100 0x7ff7f4151630 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff7f40b5ca0 0x7ff7f414d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 --2- 172.21.15.88:0/610211048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f40add80 0x7ff7f4158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 >> 172.21.15.88:0/610211048 conn(0x7ff7f40b2d50 msgr2=0x7ff7f40ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 shutdown_connections 2024-04-26T20:00:50.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.504+0000 7ff7fb400640 1 -- 172.21.15.88:0/610211048 wait complete. 2024-04-26T20:00:50.523 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:00:50.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.731+0000 7f3a8e400640 1 Processor -- start 2024-04-26T20:00:50.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.732+0000 7f3a8e400640 1 -- start start 2024-04-26T20:00:50.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.733+0000 7f3a8e400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a881632e0 0x7f3a881636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.733+0000 7f3a8e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a880b6210 0x7f3a880b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.733+0000 7f3a8e400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a880b6710 con 0x7f3a880d3110 2024-04-26T20:00:50.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.733+0000 7f3a8e400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3a880d3600 con 0x7f3a881632e0 2024-04-26T20:00:50.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.733+0000 7f3a8e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3a880d38e0 con 0x7f3a880b6210 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.734+0000 7f3a8d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a881632e0 0x7f3a881636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.734+0000 7f3a87e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f3a880d3110 0x7f3a88162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:47578/0 (socket says 172.21.15.88:47578) 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.734+0000 7f3a87e00640 1 -- 172.21.15.88:0/2891237903 learned_addr learned my addr 172.21.15.88:0/2891237903 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.734+0000 7f3a8da00640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a880b6210 0x7f3a880b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.735+0000 7f3a8d000640 1 -- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a880b6210 msgr2=0x7f3a880b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.735+0000 7f3a8d000640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a880b6210 0x7f3a880b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.735+0000 7f3a8d000640 1 -- 172.21.15.88:0/2891237903 >> v1:172.21.15.132:6789/0 conn(0x7f3a880d3110 legacy=0x7f3a88162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.735+0000 7f3a8d000640 1 -- 172.21.15.88:0/2891237903 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880d3bc0 con 0x7f3a881632e0 2024-04-26T20:00:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.736+0000 7f3a8d000640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a881632e0 0x7f3a881636c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097a0 tx=0x7f3a7802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628991c8e15f6ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a87400640 1 -- 172.21.15.88:0/2891237903 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78037070 con 0x7f3a881632e0 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a87400640 1 -- 172.21.15.88:0/2891237903 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78032440 con 0x7f3a881632e0 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a87400640 1 -- 172.21.15.88:0/2891237903 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78032810 con 0x7f3a881632e0 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 -- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a881632e0 msgr2=0x7f3a881636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a881632e0 0x7f3a881636c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097a0 tx=0x7f3a7802a180 comp rx=0 tx=0).stop 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 -- 172.21.15.88:0/2891237903 shutdown_connections 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a880b6210 0x7f3a880b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 --2- 172.21.15.88:0/2891237903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a881632e0 0x7f3a881636c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.737+0000 7f3a8e400640 1 -- 172.21.15.88:0/2891237903 >> 172.21.15.88:0/2891237903 conn(0x7f3a8809e2f0 msgr2=0x7f3a880d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.738+0000 7f3a8e400640 1 -- 172.21.15.88:0/2891237903 shutdown_connections 2024-04-26T20:00:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.738+0000 7f3a8e400640 1 -- 172.21.15.88:0/2891237903 wait complete. 2024-04-26T20:00:50.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.739+0000 7f3a8e400640 1 Processor -- start 2024-04-26T20:00:50.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.739+0000 7f3a8e400640 1 -- start start 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a880d3110 0x7f3a88149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a881632e0 0x7f3a8815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88163d80 con 0x7f3a880b6210 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88163c00 con 0x7f3a880d3110 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.740+0000 7f3a8e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3a88163f00 con 0x7f3a881632e0 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53788/0 (socket says 172.21.15.88:53788) 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 -- 172.21.15.88:0/1991036637 learned_addr learned my addr 172.21.15.88:0/1991036637 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8da00640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a881632e0 0x7f3a8815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a87e00640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a880d3110 0x7f3a88149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 -- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a881632e0 msgr2=0x7f3a8815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a881632e0 0x7f3a8815bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 -- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a880d3110 msgr2=0x7f3a88149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a880d3110 0x7f3a88149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.741+0000 7f3a8d000640 1 -- 172.21.15.88:0/1991036637 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a880c97c0 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a8d000640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a7802a1c0 tx=0x7f3a780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78041020 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a78030700 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78037040 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8815aee0 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.742+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8815b4a0 con 0x7f3a880b6210 2024-04-26T20:00:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.743+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a880693c0 con 0x7f3a880b6210 2024-04-26T20:00:50.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.744+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a78044020 con 0x7f3a880b6210 2024-04-26T20:00:50.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.746+0000 7f3a84c00640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 0x7f3a6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:50.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.746+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f3a780ac480 con 0x7f3a880b6210 2024-04-26T20:00:50.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.746+0000 7f3a87e00640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 0x7f3a6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:50.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.747+0000 7f3a87e00640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 0x7f3a6406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a88150600 tx=0x7f3a70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:50.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.750+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a7804b030 con 0x7f3a880b6210 2024-04-26T20:00:50.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.950+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a8809e8b0 con 0x7f3a880b6210 2024-04-26T20:00:50.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.950+0000 7f3a84c00640 1 -- 172.21.15.88:0/1991036637 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f3a7807a780 con 0x7f3a880b6210 2024-04-26T20:00:50.951 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.953+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 msgr2=0x7f3a6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.953+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 0x7f3a6406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3a88150600 tx=0x7f3a70009040 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.953+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 msgr2=0x7f3a88149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.953+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a7802a1c0 tx=0x7f3a780046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 shutdown_connections 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a6406c8f0 0x7f3a6406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a881632e0 0x7f3a8815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a880d3110 0x7f3a88149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 --2- 172.21.15.88:0/1991036637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a880b6210 0x7f3a88149780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 >> 172.21.15.88:0/1991036637 conn(0x7f3a8809e2f0 msgr2=0x7f3a880b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 shutdown_connections 2024-04-26T20:00:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:50.954+0000 7f3a8e400640 1 -- 172.21.15.88:0/1991036637 wait complete. 2024-04-26T20:00:50.974 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:51.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.170+0000 7f3981200640 1 Processor -- start 2024-04-26T20:00:51.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.171+0000 7f3981200640 1 -- start start 2024-04-26T20:00:51.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3981200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f397c15f3f0 0x7f397c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3981200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 0x7f397c09bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3981200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f397c0b7aa0 con 0x7f397c0b5240 2024-04-26T20:00:51.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3981200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f397c0705b0 con 0x7f397c15f3f0 2024-04-26T20:00:51.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3981200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f397c0b7d80 con 0x7f397c0b7530 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.172+0000 7f3979e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f397c15f3f0 0x7f397c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f397c15f3f0 0x7f397c09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53800/0 (socket says 172.21.15.88:53800) 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 -- 172.21.15.88:0/1781814448 learned_addr learned my addr 172.21.15.88:0/1781814448 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979400640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 0x7f397c09bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 -- 172.21.15.88:0/1781814448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 msgr2=0x7f397c09bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 0x7f397c09bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 -- 172.21.15.88:0/1781814448 >> v1:172.21.15.132:6789/0 conn(0x7f397c0b5240 legacy=0x7f397c06dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979e00640 1 -- 172.21.15.88:0/1781814448 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c0b8060 con 0x7f397c15f3f0 2024-04-26T20:00:51.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.173+0000 7f3979400640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 0x7f397c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:51.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.174+0000 7f3979e00640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c15f3f0 0x7f397c09ba50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d0a903deebff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.174+0000 7f3978a00640 1 -- 172.21.15.88:0/1781814448 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f397c15f3f0 2024-04-26T20:00:51.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3978a00640 1 -- 172.21.15.88:0/1781814448 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c032440 con 0x7f397c15f3f0 2024-04-26T20:00:51.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3978a00640 1 -- 172.21.15.88:0/1781814448 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c032810 con 0x7f397c15f3f0 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 -- 172.21.15.88:0/1781814448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c15f3f0 msgr2=0x7f397c09ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c15f3f0 0x7f397c09ba50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a210 comp rx=0 tx=0).stop 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 -- 172.21.15.88:0/1781814448 shutdown_connections 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c0b7530 0x7f397c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 --2- 172.21.15.88:0/1781814448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c15f3f0 0x7f397c09ba50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 -- 172.21.15.88:0/1781814448 >> 172.21.15.88:0/1781814448 conn(0x7f397c075f30 msgr2=0x7f397c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 -- 172.21.15.88:0/1781814448 shutdown_connections 2024-04-26T20:00:51.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.175+0000 7f3981200640 1 -- 172.21.15.88:0/1781814448 wait complete. 2024-04-26T20:00:51.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.177+0000 7f3981200640 1 Processor -- start 2024-04-26T20:00:51.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.177+0000 7f3981200640 1 -- start start 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f397c0b7530 0x7f397c152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c15f3f0 0x7f397c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f397c071bd0 con 0x7f397c0b5240 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f397c071a50 con 0x7f397c0b7530 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3981200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f397c071d50 con 0x7f397c15f3f0 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3979e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3979e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53814/0 (socket says 172.21.15.88:53814) 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.178+0000 7f3979e00640 1 -- 172.21.15.88:0/3963064735 learned_addr learned my addr 172.21.15.88:0/3963064735 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 -- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c15f3f0 msgr2=0x7f397c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c15f3f0 0x7f397c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 -- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f397c0b7530 msgr2=0x7f397c152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f397c0b7530 0x7f397c152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 -- 172.21.15.88:0/3963064735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c157b00 con 0x7f397c0b5240 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3979e00640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f396c000c00 tx=0x7f396c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037040 con 0x7f397c0b5240 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c042070 con 0x7f397c0b5240 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c030bd0 con 0x7f397c0b5240 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.179+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c157de0 con 0x7f397c0b5240 2024-04-26T20:00:51.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.180+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1583a0 con 0x7f397c0b5240 2024-04-26T20:00:51.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.181+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c069540 con 0x7f397c0b5240 2024-04-26T20:00:51.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.182+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f396c047460 con 0x7f397c0b5240 2024-04-26T20:00:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.182+0000 7f3972000640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 0x7f395c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.183+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f396c0ac8b0 con 0x7f397c0b5240 2024-04-26T20:00:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.183+0000 7f3979400640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 0x7f395c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.184+0000 7f3979400640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 0x7f395c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f397c156200 tx=0x7f39680083d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.187+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f396c07abb0 con 0x7f397c0b5240 2024-04-26T20:00:51.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.433+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f397c06fc70 con 0x7f397c0b5240 2024-04-26T20:00:51.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.435+0000 7f3972000640 1 -- 172.21.15.88:0/3963064735 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f396c07a4b0 con 0x7f397c0b5240 2024-04-26T20:00:51.435 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:51.436 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:51.436 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 msgr2=0x7f395c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 0x7f395c06edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f397c156200 tx=0x7f39680083d0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 msgr2=0x7f397c1620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f396c000c00 tx=0x7f396c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 shutdown_connections 2024-04-26T20:00:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f395c06c8f0 0x7f395c06edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f397c15f3f0 0x7f397c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f397c0b7530 0x7f397c152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 --2- 172.21.15.88:0/3963064735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f397c0b5240 0x7f397c1620b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.438+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 >> 172.21.15.88:0/3963064735 conn(0x7f397c075f30 msgr2=0x7f397c0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.439+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 shutdown_connections 2024-04-26T20:00:51.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.439+0000 7f3981200640 1 -- 172.21.15.88:0/3963064735 wait complete. 2024-04-26T20:00:51.459 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:51.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.660+0000 7f71ef400640 1 Processor -- start 2024-04-26T20:00:51.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.660+0000 7f71ef400640 1 -- start start 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71ef400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71e815cdf0 0x7f71e80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71ef400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e806ce80 0x7f71e80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71ef400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e80af9a0 con 0x7f71e80cd650 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71ef400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f71e80ae2c0 con 0x7f71e815cdf0 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71ef400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71e80b05b0 con 0x7f71e806ce80 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71eca00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71e815cdf0 0x7f71e80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71eca00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71e815cdf0 0x7f71e80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53828/0 (socket says 172.21.15.88:53828) 2024-04-26T20:00:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.662+0000 7f71eca00640 1 -- 172.21.15.88:0/2977608269 learned_addr learned my addr 172.21.15.88:0/2977608269 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:51.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.663+0000 7f71ed400640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e806ce80 0x7f71e80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.663+0000 7f71eca00640 1 -- 172.21.15.88:0/2977608269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e806ce80 msgr2=0x7f71e80b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:51.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.663+0000 7f71eca00640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e806ce80 0x7f71e80b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.663+0000 7f71eca00640 1 -- 172.21.15.88:0/2977608269 >> v1:172.21.15.132:6789/0 conn(0x7f71e80cd650 legacy=0x7f71e806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.663+0000 7f71eca00640 1 -- 172.21.15.88:0/2977608269 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e80afc80 con 0x7f71e815cdf0 2024-04-26T20:00:51.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.664+0000 7f71eca00640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e80add80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c68603f5f72332 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.665+0000 7f71e7400640 1 -- 172.21.15.88:0/2977608269 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc037070 con 0x7f71e815cdf0 2024-04-26T20:00:51.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.665+0000 7f71e7400640 1 -- 172.21.15.88:0/2977608269 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc032440 con 0x7f71e815cdf0 2024-04-26T20:00:51.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.665+0000 7f71e7400640 1 -- 172.21.15.88:0/2977608269 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc032810 con 0x7f71e815cdf0 2024-04-26T20:00:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.665+0000 7f71ef400640 1 -- 172.21.15.88:0/2977608269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 msgr2=0x7f71e80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.665+0000 7f71ef400640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e80add80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a270 comp rx=0 tx=0).stop 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 -- 172.21.15.88:0/2977608269 shutdown_connections 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e806ce80 0x7f71e80b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 --2- 172.21.15.88:0/2977608269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e80add80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 -- 172.21.15.88:0/2977608269 >> 172.21.15.88:0/2977608269 conn(0x7f71e80b2d50 msgr2=0x7f71e80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 -- 172.21.15.88:0/2977608269 shutdown_connections 2024-04-26T20:00:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.666+0000 7f71ef400640 1 -- 172.21.15.88:0/2977608269 wait complete. 2024-04-26T20:00:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.667+0000 7f71ef400640 1 Processor -- start 2024-04-26T20:00:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.668+0000 7f71ef400640 1 -- start start 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.668+0000 7f71ef400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71e806ce80 0x7f71e8152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.668+0000 7f71ef400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e80cd650 0x7f71e809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.668+0000 7f71ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ef400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71e815d1d0 con 0x7f71e815cdf0 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ef400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71e80b0240 con 0x7f71e806ce80 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ef400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71e815d350 con 0x7f71e80cd650 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53836/0 (socket says 172.21.15.88:53836) 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 -- 172.21.15.88:0/909454160 learned_addr learned my addr 172.21.15.88:0/909454160 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71eca00640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71e806ce80 0x7f71e8152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71e7e00640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e80cd650 0x7f71e809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 -- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e80cd650 msgr2=0x7f71e809da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e80cd650 0x7f71e809da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 -- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71e806ce80 msgr2=0x7f71e8152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71e806ce80 0x7f71e8152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.669+0000 7f71ed400640 1 -- 172.21.15.88:0/909454160 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e809df70 con 0x7f71e815cdf0 2024-04-26T20:00:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71ed400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009fd0 tx=0x7f71e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0014420 con 0x7f71e815cdf0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e0019070 con 0x7f71e815cdf0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0014800 con 0x7f71e815cdf0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e815e530 con 0x7f71e815cdf0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.670+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e815eaf0 con 0x7f71e815cdf0 2024-04-26T20:00:51.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.671+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e8069540 con 0x7f71e815cdf0 2024-04-26T20:00:51.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.672+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71e00149a0 con 0x7f71e815cdf0 2024-04-26T20:00:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.673+0000 7f71e4c00640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 0x7f71c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.673+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f71e008c480 con 0x7f71e815cdf0 2024-04-26T20:00:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.673+0000 7f71eca00640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 0x7f71c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.674+0000 7f71eca00640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 0x7f71c806ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:51.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.678+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71e005a780 con 0x7f71e815cdf0 2024-04-26T20:00:51.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.877+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71e80b5f10 con 0x7f71e815cdf0 2024-04-26T20:00:51.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.878+0000 7f71e4c00640 1 -- 172.21.15.88:0/909454160 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f71e005a080 con 0x7f71e815cdf0 2024-04-26T20:00:51.879 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:51.879 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:51.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 msgr2=0x7f71c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 0x7f71c806ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02e000 comp rx=0 tx=0).stop 2024-04-26T20:00:51.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 msgr2=0x7f71e815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009fd0 tx=0x7f71e000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 shutdown_connections 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71e815cdf0 0x7f71e815b820 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f71c806c920 0x7f71c806ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71e80cd650 0x7f71e809da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 --2- 172.21.15.88:0/909454160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71e806ce80 0x7f71e8152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 >> 172.21.15.88:0/909454160 conn(0x7f71e80b2d50 msgr2=0x7f71e806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.881+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 shutdown_connections 2024-04-26T20:00:51.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:51.882+0000 7f71ef400640 1 -- 172.21.15.88:0/909454160 wait complete. 2024-04-26T20:00:51.902 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:00:51.902 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.103+0000 7f462fe00640 1 Processor -- start 2024-04-26T20:00:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.103+0000 7f462fe00640 1 -- start start 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f463015f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4630073ed0 0x7f463015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462fe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4630073110 con 0x7f4630145570 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f46300cef40 con 0x7f4630073ed0 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f46301606b0 con 0x7f46300d1580 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462ea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4630145570 0x7f46300d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36060/0 (socket says 172.21.15.88:36060) 2024-04-26T20:00:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.105+0000 7f462ea00640 1 -- 172.21.15.88:0/3534091603 learned_addr learned my addr 172.21.15.88:0/3534091603 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:52.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462f400640 1 --2- 172.21.15.88:0/3534091603 >> v2:172.21.15.132:3300/0 conn(0x7f4630073ed0 0x7f463015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f4627e00640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f463015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118653426 0 0) 0x7f4630073110 con 0x7f4630145570 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4614003520 con 0x7f4630145570 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4044167612 0 0) 0x7f4614003520 con 0x7f4630145570 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 msgr2=0x7f463015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f463015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 >> v2:172.21.15.132:3300/0 conn(0x7f4630073ed0 msgr2=0x7f463015fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 --2- 172.21.15.88:0/3534091603 >> v2:172.21.15.132:3300/0 conn(0x7f4630073ed0 0x7f463015fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46300733f0 con 0x7f4630145570 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f4620001670 con 0x7f4630145570 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 >> v1:172.21.15.88:6789/0 conn(0x7f4630145570 legacy=0x7f46300d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46140039b0 0x7f4614003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f4614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.106+0000 7f462e000640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4614004d40 0x7f46140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46300cef40 con 0x7f4614004d40 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f46301606b0 con 0x7f46140039b0 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f461400dbd0 con 0x7f46300d1580 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4614004d40 0x7f46140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f4627e00640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f4614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 -- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 msgr2=0x7f4614004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f46300d1580 0x7f4614004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 -- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46140039b0 msgr2=0x7f4614003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46140039b0 0x7f4614003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.107+0000 7f462f400640 1 -- 172.21.15.88:0/3534091603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614003520 con 0x7f4614004d40 2024-04-26T20:00:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.108+0000 7f462f400640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4614004d40 0x7f46140051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fc0 tx=0x7f4614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83783a12867a933f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628009c10 con 0x7f4614004d40 2024-04-26T20:00:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4628036070 con 0x7f4614004d40 2024-04-26T20:00:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462e000640 1 -- 172.21.15.88:0/3534091603 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462802d070 con 0x7f4614004d40 2024-04-26T20:00:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 -- 172.21.15.88:0/3534091603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4614004d40 msgr2=0x7f46140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 --2- 172.21.15.88:0/3534091603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4614004d40 0x7f46140051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fc0 tx=0x7f4614005830 comp rx=0 tx=0).stop 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462f400640 1 -- 172.21.15.88:0/3534091603 reap_dead start 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 -- 172.21.15.88:0/3534091603 shutdown_connections 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 -- 172.21.15.88:0/3534091603 >> 172.21.15.88:0/3534091603 conn(0x7f463006c740 msgr2=0x7f463006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 -- 172.21.15.88:0/3534091603 shutdown_connections 2024-04-26T20:00:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.109+0000 7f462fe00640 1 -- 172.21.15.88:0/3534091603 wait complete. 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.111+0000 7f462fe00640 1 Processor -- start 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.111+0000 7f462fe00640 1 -- start start 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4614004d40 0x7f46300c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46300c8e00 0x7f46300c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46301603c0 con 0x7f46300c6410 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f46300739b0 con 0x7f46300c8e00 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.112+0000 7f462fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4630145950 con 0x7f4614004d40 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53850/0 (socket says 172.21.15.88:53850) 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 -- 172.21.15.88:0/562407301 learned_addr learned my addr 172.21.15.88:0/562407301 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f4627e00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46300c8e00 0x7f46300c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462ea00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4614004d40 0x7f46300c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 -- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4614004d40 msgr2=0x7f46300c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4614004d40 0x7f46300c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 -- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46300c8e00 msgr2=0x7f46300c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46300c8e00 0x7f46300c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.113+0000 7f462f400640 1 -- 172.21.15.88:0/562407301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46300c68b0 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f462f400640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4628033040 tx=0x7f4628009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628009c10 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4628003990 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46300c6b90 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628003d60 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.114+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46300c7150 con 0x7f46300c6410 2024-04-26T20:00:52.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.115+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46300693c0 con 0x7f46300c6410 2024-04-26T20:00:52.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.116+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4628005d10 con 0x7f46300c6410 2024-04-26T20:00:52.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.117+0000 7f4626a00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 0x7f460c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.117+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f46280ad2b0 con 0x7f46300c6410 2024-04-26T20:00:52.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.117+0000 7f462ea00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 0x7f460c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.118+0000 7f462ea00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 0x7f460c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4620005af0 tx=0x7f46200016a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.122+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f462804a060 con 0x7f46300c6410 2024-04-26T20:00:52.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.370+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46300b4850 con 0x7f46300c6410 2024-04-26T20:00:52.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.372+0000 7f4626a00640 1 -- 172.21.15.88:0/562407301 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f462807b5b0 con 0x7f46300c6410 2024-04-26T20:00:52.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:52.373 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:52.373 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 msgr2=0x7f460c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 0x7f460c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4620005af0 tx=0x7f46200016a0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 msgr2=0x7f46301421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4628033040 tx=0x7f4628009750 comp rx=0 tx=0).stop 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 shutdown_connections 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46300c6410 0x7f46301421c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46300c8e00 0x7f46300c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f460c06c920 0x7f460c06ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 --2- 172.21.15.88:0/562407301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4614004d40 0x7f46300c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 >> 172.21.15.88:0/562407301 conn(0x7f463006c740 msgr2=0x7f4630144850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:52.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 shutdown_connections 2024-04-26T20:00:52.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.375+0000 7f462fe00640 1 -- 172.21.15.88:0/562407301 wait complete. 2024-04-26T20:00:52.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:00:52.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.599+0000 7f9ed7c00640 1 Processor -- start 2024-04-26T20:00:52.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.600+0000 7f9ed7c00640 1 -- start start 2024-04-26T20:00:52.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.601+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9ed006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.601+0000 7f9ed7c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9ed015cb70 0x7f9ed0073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.601+0000 7f9ed7c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed006d090 con 0x7f9ed015cfd0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.601+0000 7f9ed7c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed00af460 con 0x7f9ed015cb70 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.601+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed00ae850 con 0x7f9ed006c770 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ed5c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9ed015cfd0 0x7f9ed0073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36068/0 (socket says 172.21.15.88:36068) 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3846376805 learned_addr learned my addr 172.21.15.88:0/3846376805 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ed5200640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9ed006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3846376805 >> v2:172.21.15.132:3300/0 conn(0x7f9ed015cb70 0x7f9ed0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471055441 0 0) 0x7f9ed006d090 con 0x7f9ed015cfd0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb8005730 con 0x7f9ed015cfd0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 908852481 0 0) 0x7f9eb8005730 con 0x7f9ed015cfd0 2024-04-26T20:00:52.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 msgr2=0x7f9ed006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.602+0000 7f9ecf400640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9ed006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 >> v2:172.21.15.132:3300/0 conn(0x7f9ed015cb70 msgr2=0x7f9ed0073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 --2- 172.21.15.88:0/3846376805 >> v2:172.21.15.132:3300/0 conn(0x7f9ed015cb70 0x7f9ed0073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed00aeb30 con 0x7f9ed015cfd0 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f9ec80019f0 con 0x7f9ed015cfd0 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 >> v1:172.21.15.88:6789/0 conn(0x7f9ed015cfd0 legacy=0x7f9ed0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb8005bc0 0x7f9eb8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb8006a10 0x7f9eb8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9eb80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00af460 con 0x7f9eb8006a10 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00ae850 con 0x7f9eb8005bc0 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb800dbd0 con 0x7f9ed006c770 2024-04-26T20:00:52.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb8006a10 0x7f9eb8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ed5c00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9eb80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 -- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 msgr2=0x7f9eb80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9eb80073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 -- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb8005bc0 msgr2=0x7f9eb8005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb8005bc0 0x7f9eb8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ecfe00640 1 -- 172.21.15.88:0/3846376805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb8005730 con 0x7f9eb8006a10 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.603+0000 7f9ed5c00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9ed006c770 0x7f9eb80073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.604+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb8006a10 0x7f9eb8006e90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009ac0 tx=0x7f9eb8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b0aa37fa2e9e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc036070 con 0x7f9eb8006a10 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc031400 con 0x7f9eb8006a10 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ecf400640 1 -- 172.21.15.88:0/3846376805 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc009690 con 0x7f9eb8006a10 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3846376805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb8006a10 msgr2=0x7f9eb8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3846376805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb8006a10 0x7f9eb8006e90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009ac0 tx=0x7f9eb8007a40 comp rx=0 tx=0).stop 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3846376805 reap_dead start 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.605+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3846376805 shutdown_connections 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.606+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3846376805 >> 172.21.15.88:0/3846376805 conn(0x7f9ed00b2d50 msgr2=0x7f9ed00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.606+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3846376805 shutdown_connections 2024-04-26T20:00:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.606+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3846376805 wait complete. 2024-04-26T20:00:52.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.607+0000 7f9ed7c00640 1 Processor -- start 2024-04-26T20:00:52.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.608+0000 7f9ed7c00640 1 -- start start 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eb8006a10 0x7f9ed009e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 0x7f9ed00caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed015c600 con 0x7f9ed0154bc0 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed00af0f0 con 0x7f9ed009ba50 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed015c780 con 0x7f9eb8006a10 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53872/0 (socket says 172.21.15.88:53872) 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3405170983 learned_addr learned my addr 172.21.15.88:0/3405170983 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5200640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eb8006a10 0x7f9ed009e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 0x7f9ed00caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eb8006a10 msgr2=0x7f9ed009e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eb8006a10 0x7f9ed009e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 msgr2=0x7f9ed00caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 0x7f9ed00caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ed5c00640 1 -- 172.21.15.88:0/3405170983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed00ca290 con 0x7f9ed0154bc0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.609+0000 7f9ecfe00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 0x7f9ed00caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9ed5c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00b3900 tx=0x7f9ec800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800bb80 con 0x7f9ed0154bc0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec800bd20 con 0x7f9ed0154bc0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed009e930 con 0x7f9ed0154bc0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007290 con 0x7f9ed0154bc0 2024-04-26T20:00:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.610+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed0153e00 con 0x7f9ed0154bc0 2024-04-26T20:00:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.611+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed0069540 con 0x7f9ed0154bc0 2024-04-26T20:00:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.612+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec801b020 con 0x7f9ed0154bc0 2024-04-26T20:00:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.613+0000 7f9eccc00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.613+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f9ec808be50 con 0x7f9ed0154bc0 2024-04-26T20:00:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.613+0000 7f9ed5200640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.614+0000 7f9ed5200640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4004770 tx=0x7f9ec4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:52.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.618+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ec805a1f0 con 0x7f9ed0154bc0 2024-04-26T20:00:52.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.861+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ed00cd430 con 0x7f9ed0154bc0 2024-04-26T20:00:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.862+0000 7f9eccc00640 1 -- 172.21.15.88:0/3405170983 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5181 (secure 0 0 0) 0x7f9ec8059af0 con 0x7f9ed0154bc0 2024-04-26T20:00:52.863 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:52.863 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3152198353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3152198353},{"type":"v1","addr":"172.21.15.132:6833","nonce":3152198353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3067516770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":3067516770},{"type":"v1","addr":"172.21.15.132:6835","nonce":3067516770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2220725413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2220725413},{"type":"v1","addr":"172.21.15.88:6836","nonce":2220725413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:59:45.722094+0000","modified":"2024-04-26T19:59:48.769966+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/3248668459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":3248668459},{"type":"v1","addr":"172.21.15.88:6837","nonce":3248668459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-26T20:00:52.863 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 7 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 msgr2=0x7f9eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4004770 tx=0x7f9ec4008040 comp rx=0 tx=0).stop 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 msgr2=0x7f9ed0155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9ed00b3900 tx=0x7f9ec800b4a0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 shutdown_connections 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ed0154bc0 0x7f9ed0155020 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ed009ba50 0x7f9ed00caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 --2- 172.21.15.88:0/3405170983 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9eb8006a10 0x7f9ed009e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 >> 172.21.15.88:0/3405170983 conn(0x7f9ed00b2d50 msgr2=0x7f9ed00b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 shutdown_connections 2024-04-26T20:00:52.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:52.865+0000 7f9ed7c00640 1 -- 172.21.15.88:0/3405170983 wait complete. 2024-04-26T20:00:52.886 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:00:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.080+0000 7f1ab3200640 1 Processor -- start 2024-04-26T20:00:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.080+0000 7f1ab3200640 1 -- start start 2024-04-26T20:00:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.081+0000 7f1ab3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1ab40d2bf0 0x7f1ab40d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.081+0000 7f1ab3200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab40d04a0 con 0x7f1ab40710d0 2024-04-26T20:00:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.081+0000 7f1ab3200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab40cfee0 con 0x7f1ab40b2200 2024-04-26T20:00:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.081+0000 7f1ab3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab406e050 con 0x7f1ab40d2bf0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab1400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1ab40710d0 0x7f1ab406e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36084/0 (socket says 172.21.15.88:36084) 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab1400640 1 -- 172.21.15.88:0/3233540725 learned_addr learned my addr 172.21.15.88:0/3233540725 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710238262 0 0) 0x7f1ab40d04a0 con 0x7f1ab40710d0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a98003520 con 0x7f1ab40710d0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab2800640 1 --2- 172.21.15.88:0/3233540725 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40d2bf0 0x7f1ab40d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3765981150 0 0) 0x7f1a98003520 con 0x7f1ab40710d0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40d2bf0 msgr2=0x7f1ab40d3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 --2- 172.21.15.88:0/3233540725 >> v2:172.21.15.132:3300/0 conn(0x7f1ab40d2bf0 0x7f1ab40d3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 >> v1:172.21.15.132:6789/0 conn(0x7f1ab40b2200 legacy=0x7f1ab40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40d01c0 con 0x7f1ab40710d0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f1a9c001670 con 0x7f1ab40710d0 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 >> v1:172.21.15.88:6789/0 conn(0x7f1ab40710d0 legacy=0x7f1ab406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980039b0 0x7f1a98003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a98004800 0x7f1a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 0x7f1a9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab406e050 con 0x7f1a980051c0 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9800c0f0 con 0x7f1a980039b0 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.082+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9800c2f0 con 0x7f1a98004800 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 0x7f1a9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab1e00640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980039b0 0x7f1a98003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab1400640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a98004800 0x7f1a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 -- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a98004800 msgr2=0x7f1a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a98004800 0x7f1a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 -- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980039b0 msgr2=0x7f1a98003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980039b0 0x7f1a98003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.083+0000 7f1ab2800640 1 -- 172.21.15.88:0/3233540725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a98003520 con 0x7f1a980051c0 2024-04-26T20:00:53.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.084+0000 7f1ab2800640 1 -- 172.21.15.88:0/3233540725 reap_dead start 2024-04-26T20:00:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.084+0000 7f1ab2800640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 0x7f1a9800b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007330 tx=0x7f1a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af293d9bbc75224f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.084+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009be0 con 0x7f1a980051c0 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.084+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac003b90 con 0x7f1a980051c0 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.084+0000 7f1ab0a00640 1 -- 172.21.15.88:0/3233540725 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004020 con 0x7f1a980051c0 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 -- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 msgr2=0x7f1a9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 0x7f1a9800b9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1aac007330 tx=0x7f1a98005830 comp rx=0 tx=0).stop 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 -- 172.21.15.88:0/3233540725 shutdown_connections 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 --2- 172.21.15.88:0/3233540725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a980051c0 0x7f1a9800b9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 -- 172.21.15.88:0/3233540725 >> 172.21.15.88:0/3233540725 conn(0x7f1ab409ba50 msgr2=0x7f1ab409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 -- 172.21.15.88:0/3233540725 shutdown_connections 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.085+0000 7f1ab3200640 1 -- 172.21.15.88:0/3233540725 wait complete. 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.087+0000 7f1ab3200640 1 Processor -- start 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.087+0000 7f1ab3200640 1 -- start start 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980051c0 0x7f1ab40c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4161270 0x7f1ab41f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40d3200 con 0x7f1ab41f5550 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40d3080 con 0x7f1a980051c0 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40d3380 con 0x7f1ab4161270 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53898/0 (socket says 172.21.15.88:53898) 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.088+0000 7f1ab2800640 1 -- 172.21.15.88:0/643265327 learned_addr learned my addr 172.21.15.88:0/643265327 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab1e00640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980051c0 0x7f1ab40c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab1400640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4161270 0x7f1ab41f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 -- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4161270 msgr2=0x7f1ab41f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4161270 0x7f1ab41f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 -- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980051c0 msgr2=0x7f1ab40c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980051c0 0x7f1ab40c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 -- 172.21.15.88:0/643265327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40b6bb0 con 0x7f1ab41f5550 2024-04-26T20:00:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1ab2800640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1aac02b070 tx=0x7f1aac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac046070 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac003990 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.089+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009be0 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.090+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab40b6e90 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.090+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab40b7450 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.091+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab40693c0 con 0x7f1ab41f5550 2024-04-26T20:00:53.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.092+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aac049070 con 0x7f1ab41f5550 2024-04-26T20:00:53.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.093+0000 7f1aaa000640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 0x7f1a9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.093+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7f1aac0ac000 con 0x7f1ab41f5550 2024-04-26T20:00:53.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.093+0000 7f1ab1e00640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 0x7f1a9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.094+0000 7f1ab1e00640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 0x7f1a9006ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4004dc0 tx=0x7f1aa4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.097+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1aac032030 con 0x7f1ab41f5550 2024-04-26T20:00:53.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.280+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ab40d04a0 con 0x7f1ab41f5550 2024-04-26T20:00:53.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.284+0000 7f1aaa000640 1 -- 172.21.15.88:0/643265327 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16889 (secure 0 0 0) 0x7f1aac003b60 con 0x7f1ab41f5550 2024-04-26T20:00:53.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:53.285 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:28.119660+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:59:44.098424+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:59:44.695841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:00:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 msgr2=0x7f1a9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 0x7f1a9006ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4004dc0 tx=0x7f1aa4009040 comp rx=0 tx=0).stop 2024-04-26T20:00:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 msgr2=0x7f1ab40b7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1aac02b070 tx=0x7f1aac004ab0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 shutdown_connections 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ab41f5550 0x7f1ab40b7c50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.286+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a9006c920 0x7f1a9006ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.287+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1ab4161270 0x7f1ab41f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.287+0000 7f1ab3200640 1 --2- 172.21.15.88:0/643265327 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a980051c0 0x7f1ab40c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.287+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 >> 172.21.15.88:0/643265327 conn(0x7f1ab409ba50 msgr2=0x7f1ab40b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.287+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 shutdown_connections 2024-04-26T20:00:53.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.287+0000 7f1ab3200640 1 -- 172.21.15.88:0/643265327 wait complete. 2024-04-26T20:00:53.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:00:53.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.503+0000 7fde75000640 1 Processor -- start 2024-04-26T20:00:53.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.504+0000 7fde75000640 1 -- start start 2024-04-26T20:00:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.505+0000 7fde75000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fde700ca530 0x7fde70169f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.505+0000 7fde75000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7015f800 con 0x7fde700d3680 2024-04-26T20:00:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.505+0000 7fde75000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde700d2ec0 con 0x7fde7015f420 2024-04-26T20:00:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.505+0000 7fde75000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fde700d2c70 con 0x7fde700ca530 2024-04-26T20:00:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6d400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fde700d3680 0x7fde700d3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36098/0 (socket says 172.21.15.88:36098) 2024-04-26T20:00:53.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6d400640 1 -- 172.21.15.88:0/3604949597 learned_addr learned my addr 172.21.15.88:0/3604949597 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:53.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6e800640 1 --2- 172.21.15.88:0/3604949597 >> v2:172.21.15.88:3300/0 conn(0x7fde700ca530 0x7fde70169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6ca00640 1 -- 172.21.15.88:0/3604949597 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 484577700 0 0) 0x7fde7015f800 con 0x7fde700d3680 2024-04-26T20:00:53.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6ca00640 1 -- 172.21.15.88:0/3604949597 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde54003520 con 0x7fde700d3680 2024-04-26T20:00:53.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6e800640 1 -- 172.21.15.88:0/3604949597 >> v1:172.21.15.132:6789/0 conn(0x7fde7015f420 legacy=0x7fde700d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6e800640 1 -- 172.21.15.88:0/3604949597 >> v1:172.21.15.88:6789/0 conn(0x7fde700d3680 legacy=0x7fde700d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.506+0000 7fde6e800640 1 -- 172.21.15.88:0/3604949597 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde700d31a0 con 0x7fde700ca530 2024-04-26T20:00:53.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.508+0000 7fde6e800640 1 --2- 172.21.15.88:0/3604949597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde70169f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde680097a0 tx=0x7fde6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e905644b1e0ad23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.508+0000 7fde6ca00640 1 -- 172.21.15.88:0/3604949597 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68037070 con 0x7fde700ca530 2024-04-26T20:00:53.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.508+0000 7fde6ca00640 1 -- 172.21.15.88:0/3604949597 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde68032440 con 0x7fde700ca530 2024-04-26T20:00:53.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.508+0000 7fde6ca00640 1 -- 172.21.15.88:0/3604949597 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68032810 con 0x7fde700ca530 2024-04-26T20:00:53.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 -- 172.21.15.88:0/3604949597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 msgr2=0x7fde70169f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 --2- 172.21.15.88:0/3604949597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde70169f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde680097a0 tx=0x7fde6802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 -- 172.21.15.88:0/3604949597 shutdown_connections 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 --2- 172.21.15.88:0/3604949597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde70169f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 -- 172.21.15.88:0/3604949597 >> 172.21.15.88:0/3604949597 conn(0x7fde7009e3e0 msgr2=0x7fde700d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 -- 172.21.15.88:0/3604949597 shutdown_connections 2024-04-26T20:00:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.509+0000 7fde75000640 1 -- 172.21.15.88:0/3604949597 wait complete. 2024-04-26T20:00:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.510+0000 7fde75000640 1 Processor -- start 2024-04-26T20:00:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.511+0000 7fde75000640 1 -- start start 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.511+0000 7fde75000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde75000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 0x7fde700ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde75000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde7015f420 0x7fde700ba810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde75000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde700c9000 con 0x7fde700ca530 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde75000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fde700d3480 con 0x7fde700d3680 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde75000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fde700d0f30 con 0x7fde7015f420 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53920/0 (socket says 172.21.15.88:53920) 2024-04-26T20:00:53.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 -- 172.21.15.88:0/438963389 learned_addr learned my addr 172.21.15.88:0/438963389 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6d400640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 0x7fde700ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 -- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde7015f420 msgr2=0x7fde700ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde7015f420 0x7fde700ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 -- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 msgr2=0x7fde700ba2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 0x7fde700ba2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6de00640 1 -- 172.21.15.88:0/438963389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde70152a30 con 0x7fde700ca530 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.512+0000 7fde6d400640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 0x7fde700ba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:53.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde6de00640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c870 tx=0x7fde6000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60003b70 con 0x7fde700ca530 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde60003d30 con 0x7fde700ca530 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6000aa90 con 0x7fde700ca530 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde701f4ec0 con 0x7fde700ca530 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.513+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde700b8bc0 con 0x7fde700ca530 2024-04-26T20:00:53.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.514+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde70069540 con 0x7fde700ca530 2024-04-26T20:00:53.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.515+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde6000ac30 con 0x7fde700ca530 2024-04-26T20:00:53.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.516+0000 7fde66000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 0x7fde4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.516+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5854+0+0 (secure 0 0 0) 0x7fde6008bc40 con 0x7fde700ca530 2024-04-26T20:00:53.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.516+0000 7fde6d400640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 0x7fde4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.517+0000 7fde6d400640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 0x7fde4c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde701627f0 tx=0x7fde58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.520+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde60059f40 con 0x7fde700ca530 2024-04-26T20:00:53.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.743+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fde700d2150 con 0x7fde700ca530 2024-04-26T20:00:53.745 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-04-26T20:00:53.744+0000 7fad57a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:00:53.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.748+0000 7fde66000640 1 -- 172.21.15.88:0/438963389 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fde60059840 con 0x7fde700ca530 2024-04-26T20:00:53.749 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.751+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 msgr2=0x7fde4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.751+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 0x7fde4c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde701627f0 tx=0x7fde58009040 comp rx=0 tx=0).stop 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.751+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 msgr2=0x7fde701524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.751+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fde6000c870 tx=0x7fde6000cd30 comp rx=0 tx=0).stop 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 shutdown_connections 2024-04-26T20:00:53.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fde4c06c8f0 0x7fde4c06edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fde7015f420 0x7fde700ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde700d3680 0x7fde700ba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 --2- 172.21.15.88:0/438963389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde700ca530 0x7fde701524f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.752+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 >> 172.21.15.88:0/438963389 conn(0x7fde7009e3e0 msgr2=0x7fde700caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.753+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 shutdown_connections 2024-04-26T20:00:53.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.753+0000 7fde75000640 1 -- 172.21.15.88:0/438963389 wait complete. 2024-04-26T20:00:53.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:00:53.777 INFO:tasks.ceph.mds.a.smithi088.stderr: -4202> 2024-04-26T19:59:47.227+0000 7f32f9600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:00:53.837 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:00:53.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.946+0000 7f71b7c00640 1 Processor -- start 2024-04-26T20:00:53.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.946+0000 7f71b7c00640 1 -- start start 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 0x7f71b006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71b00b5ca0 0x7f71b006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f71b00add80 0x7f71b0072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f71b00af800 con 0x7f71b00b5ca0 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f71b00ae340 con 0x7f71b00add80 2024-04-26T20:00:53.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71b7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71b00afaa0 con 0x7f71b015d100 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71a7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71b00b5ca0 0x7f71b006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.948+0000 7f71a7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f71b00b5ca0 0x7f71b006ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53952/0 (socket says 172.21.15.88:53952) 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 -- 172.21.15.88:0/195357875 learned_addr learned my addr 172.21.15.88:0/195357875 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71b5200640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 0x7f71b006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71b5c00640 1 --2- 172.21.15.88:0/195357875 >> v2:172.21.15.132:3300/0 conn(0x7f71b00add80 0x7f71b0072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 -- 172.21.15.88:0/195357875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 msgr2=0x7f71b006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 0x7f71b006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 -- 172.21.15.88:0/195357875 >> v2:172.21.15.132:3300/0 conn(0x7f71b00add80 msgr2=0x7f71b0072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 --2- 172.21.15.88:0/195357875 >> v2:172.21.15.132:3300/0 conn(0x7f71b00add80 0x7f71b0072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71a7e00640 1 -- 172.21.15.88:0/195357875 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b00afd80 con 0x7f71b00b5ca0 2024-04-26T20:00:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.949+0000 7f71b5200640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 0x7f71b006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.950+0000 7f71a7e00640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00b5ca0 0x7f71b006ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71b0073c70 tx=0x7f719802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be8931994833efc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.950+0000 7f71afe00640 1 -- 172.21.15.88:0/195357875 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198037070 con 0x7f71b00b5ca0 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.950+0000 7f71afe00640 1 -- 172.21.15.88:0/195357875 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198032440 con 0x7f71b00b5ca0 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.950+0000 7f71afe00640 1 -- 172.21.15.88:0/195357875 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198032810 con 0x7f71b00b5ca0 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 -- 172.21.15.88:0/195357875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00b5ca0 msgr2=0x7f71b006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00b5ca0 0x7f71b006ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71b0073c70 tx=0x7f719802a140 comp rx=0 tx=0).stop 2024-04-26T20:00:53.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 -- 172.21.15.88:0/195357875 shutdown_connections 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 --2- 172.21.15.88:0/195357875 >> v2:172.21.15.132:3300/0 conn(0x7f71b00add80 0x7f71b0072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00b5ca0 0x7f71b006ccb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 --2- 172.21.15.88:0/195357875 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b015d100 0x7f71b006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 -- 172.21.15.88:0/195357875 >> 172.21.15.88:0/195357875 conn(0x7f71b00b2d50 msgr2=0x7f71b00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 -- 172.21.15.88:0/195357875 shutdown_connections 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.951+0000 7f71b7c00640 1 -- 172.21.15.88:0/195357875 wait complete. 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.953+0000 7f71b7c00640 1 Processor -- start 2024-04-26T20:00:53.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.953+0000 7f71b7c00640 1 -- start start 2024-04-26T20:00:53.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b00b5ca0 0x7f71b015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71b015d100 0x7f71b015f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71b009ee50 con 0x7f71b00add80 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f71b00b0340 con 0x7f71b015d100 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f71b0160b70 con 0x7f71b00b5ca0 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53958/0 (socket says 172.21.15.88:53958) 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.954+0000 7f71b5200640 1 -- 172.21.15.88:0/3230325482 learned_addr learned my addr 172.21.15.88:0/3230325482 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71b015d100 0x7f71b015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71a7e00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b00b5ca0 0x7f71b015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 -- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b00b5ca0 msgr2=0x7f71b015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b00b5ca0 0x7f71b015f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 -- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71b015d100 msgr2=0x7f71b015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71b015d100 0x7f71b015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 -- 172.21.15.88:0/3230325482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01599f0 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b5200640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c820 tx=0x7f71a000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0015070 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a0007a30 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0007de0 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b015b240 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.955+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b0159650 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.957+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b0069650 con 0x7f71b00add80 2024-04-26T20:00:53.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.958+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71a0018070 con 0x7f71b00add80 2024-04-26T20:00:53.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.958+0000 7f71ad600640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 0x7f718c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:53.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.959+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5940+0+0 (secure 0 0 0) 0x7f71a008ba20 con 0x7f71b00add80 2024-04-26T20:00:53.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.959+0000 7f71a7e00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 0x7f718c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:53.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.959+0000 7f71a7e00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 0x7f718c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7198034000 tx=0x7f719802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:53.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:53.963+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71a0059df0 con 0x7f71b00add80 2024-04-26T20:00:54.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.196+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f71b0073d00 con 0x7f71b00add80 2024-04-26T20:00:54.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.752+0000 7f71ad600640 1 -- 172.21.15.88:0/3230325482 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f71a00596f0 con 0x7f71b00add80 2024-04-26T20:00:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.755+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 msgr2=0x7f718c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.755+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 0x7f718c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7198034000 tx=0x7f719802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 msgr2=0x7f71b014c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c820 tx=0x7f71a000cce0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 shutdown_connections 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71b015d100 0x7f71b015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f718c06c8f0 0x7f718c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f71b00b5ca0 0x7f71b015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 --2- 172.21.15.88:0/3230325482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71b00add80 0x7f71b014c890 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 >> 172.21.15.88:0/3230325482 conn(0x7f71b00b2d50 msgr2=0x7f71b00b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 shutdown_connections 2024-04-26T20:00:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.756+0000 7f71b7c00640 1 -- 172.21.15.88:0/3230325482 wait complete. 2024-04-26T20:00:54.777 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:00:54.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.981+0000 7f0f1c800640 1 Processor -- start 2024-04-26T20:00:54.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.981+0000 7f0f1c800640 1 -- start start 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.982+0000 7f0f1c800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0f140cdf20 0x7f0f140ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.982+0000 7f0f1c800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0f140b5740 0x7f0f140d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f1c800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f1415d820 con 0x7f0f140d0060 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f1c800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140b5b90 con 0x7f0f140cdf20 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f1c800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f0f140d0a00 con 0x7f0f140b5740 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0f140cdf20 0x7f0f140ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0f140d0060 0x7f0f140b5030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36112/0 (socket says 172.21.15.88:36112) 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19400640 1 -- 172.21.15.88:0/2186675585 learned_addr learned my addr 172.21.15.88:0/2186675585 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19e00640 1 -- 172.21.15.88:0/2186675585 >> v2:172.21.15.132:3300/0 conn(0x7f0f140b5740 msgr2=0x7f0f140d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19e00640 1 --2- 172.21.15.88:0/2186675585 >> v2:172.21.15.132:3300/0 conn(0x7f0f140b5740 0x7f0f140d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19e00640 1 -- 172.21.15.88:0/2186675585 >> v1:172.21.15.88:6789/0 conn(0x7f0f140d0060 legacy=0x7f0f140b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.983+0000 7f0f19e00640 1 -- 172.21.15.88:0/2186675585 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1415db00 con 0x7f0f140cdf20 2024-04-26T20:00:54.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f19e00640 1 --2- 172.21.15.88:0/2186675585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f140ceb80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d1aa0 tx=0x7f0f0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a8691bba780efd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:54.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f18a00640 1 -- 172.21.15.88:0/2186675585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08037070 con 0x7f0f140cdf20 2024-04-26T20:00:54.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f18a00640 1 -- 172.21.15.88:0/2186675585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f08032440 con 0x7f0f140cdf20 2024-04-26T20:00:54.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f18a00640 1 -- 172.21.15.88:0/2186675585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08032810 con 0x7f0f140cdf20 2024-04-26T20:00:54.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f1c800640 1 -- 172.21.15.88:0/2186675585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 msgr2=0x7f0f140ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.985+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2186675585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f140ceb80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0f140d1aa0 tx=0x7f0f0802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 -- 172.21.15.88:0/2186675585 shutdown_connections 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2186675585 >> v2:172.21.15.132:3300/0 conn(0x7f0f140b5740 0x7f0f140d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2186675585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f140ceb80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 -- 172.21.15.88:0/2186675585 >> 172.21.15.88:0/2186675585 conn(0x7f0f140d0c80 msgr2=0x7f0f140d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 -- 172.21.15.88:0/2186675585 shutdown_connections 2024-04-26T20:00:54.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.986+0000 7f0f1c800640 1 -- 172.21.15.88:0/2186675585 wait complete. 2024-04-26T20:00:54.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.987+0000 7f0f1c800640 1 Processor -- start 2024-04-26T20:00:54.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 -- start start 2024-04-26T20:00:54.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f140b5740 0x7f0f140ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 0x7f0f14152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f140b3e70 con 0x7f0f140cdf20 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1415e0c0 con 0x7f0f140d0060 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.988+0000 7f0f1c800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0f140b3ff0 con 0x7f0f140b5740 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53976/0 (socket says 172.21.15.88:53976) 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 -- 172.21.15.88:0/2923720084 learned_addr learned my addr 172.21.15.88:0/2923720084 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f1a800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 0x7f0f14152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19e00640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f140b5740 0x7f0f140ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 -- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f140b5740 msgr2=0x7f0f140ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f140b5740 0x7f0f140ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 -- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 msgr2=0x7f0f14152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 0x7f0f14152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 -- 172.21.15.88:0/2923720084 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f14160620 con 0x7f0f140cdf20 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f1a800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 0x7f0f14152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:54.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.989+0000 7f0f19400640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c3f0 tx=0x7f0f0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.990+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04017070 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.990+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f04005ce0 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.990+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040072f0 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.990+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141f9820 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.990+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141586e0 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.991+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14069540 con 0x7f0f140cdf20 2024-04-26T20:00:54.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.992+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f0401a070 con 0x7f0f140cdf20 2024-04-26T20:00:54.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.993+0000 7f0f0e000640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 0x7f0ef806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:54.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.993+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5940+0+0 (secure 0 0 0) 0x7f0f0408afd0 con 0x7f0f140cdf20 2024-04-26T20:00:54.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.993+0000 7f0f19e00640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 0x7f0ef806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:54.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.994+0000 7f0f19e00640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 0x7f0ef806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f08034000 tx=0x7f0f0802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:54.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:54.997+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f04059300 con 0x7f0f140cdf20 2024-04-26T20:00:55.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.200+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0f140b20b0 con 0x7f0f140cdf20 2024-04-26T20:00:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.761+0000 7f0f0e000640 1 -- 172.21.15.88:0/2923720084 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f0f04058c00 con 0x7f0f140cdf20 2024-04-26T20:00:55.763 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:00:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.765+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 msgr2=0x7f0ef806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.765+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 0x7f0ef806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f08034000 tx=0x7f0f0802e000 comp rx=0 tx=0).stop 2024-04-26T20:00:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.765+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 msgr2=0x7f0f141520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:55.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.765+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400c3f0 tx=0x7f0f0400c8b0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 shutdown_connections 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f140d0060 0x7f0f14152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f140cdf20 0x7f0f141520f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0ef806c8f0 0x7f0ef806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 --2- 172.21.15.88:0/2923720084 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0f140b5740 0x7f0f140ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 >> 172.21.15.88:0/2923720084 conn(0x7f0f140d0c80 msgr2=0x7f0f140b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:55.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.766+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 shutdown_connections 2024-04-26T20:00:55.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.767+0000 7f0f1c800640 1 -- 172.21.15.88:0/2923720084 wait complete. 2024-04-26T20:00:55.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:00:55.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.985+0000 7f6ef1600640 1 Processor -- start 2024-04-26T20:00:55.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.985+0000 7f6ef1600640 1 -- start start 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6eec143470 0x7f6eec06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143850 0x7f6eec0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6eec0cb6e0 0x7f6eec0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f6eec06df60 con 0x7f6eec143470 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f6eec146ba0 con 0x7f6eec0cb6e0 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6ef1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6eec06e200 con 0x7f6eec143850 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6eebe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6eec143470 0x7f6eec06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6eebe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6eec143470 0x7f6eec06fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53982/0 (socket says 172.21.15.88:53982) 2024-04-26T20:00:55.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.987+0000 7f6eebe00640 1 -- 172.21.15.88:0/2818218925 learned_addr learned my addr 172.21.15.88:0/2818218925 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6eebe00640 1 -- 172.21.15.88:0/2818218925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143850 msgr2=0x7f6eec0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2818218925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143850 0x7f6eec0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6eebe00640 1 -- 172.21.15.88:0/2818218925 >> v2:172.21.15.132:3300/0 conn(0x7f6eec0cb6e0 msgr2=0x7f6eec0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6ef0c00640 1 --2- 172.21.15.88:0/2818218925 >> v2:172.21.15.132:3300/0 conn(0x7f6eec0cb6e0 0x7f6eec0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2818218925 >> v2:172.21.15.132:3300/0 conn(0x7f6eec0cb6e0 0x7f6eec0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.988+0000 7f6eebe00640 1 -- 172.21.15.88:0/2818218925 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec06e4e0 con 0x7f6eec143470 2024-04-26T20:00:55.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.989+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2818218925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143470 0x7f6eec06fb10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6eec06f580 tx=0x7f6edc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1601fbc26067a5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.989+0000 7f6eeb400640 1 -- 172.21.15.88:0/2818218925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc037070 con 0x7f6eec143470 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.989+0000 7f6eeb400640 1 -- 172.21.15.88:0/2818218925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc032440 con 0x7f6eec143470 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.989+0000 7f6eeb400640 1 -- 172.21.15.88:0/2818218925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc032810 con 0x7f6eec143470 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 -- 172.21.15.88:0/2818218925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143470 msgr2=0x7f6eec06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2818218925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143470 0x7f6eec06fb10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6eec06f580 tx=0x7f6edc02a140 comp rx=0 tx=0).stop 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 -- 172.21.15.88:0/2818218925 shutdown_connections 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2818218925 >> v2:172.21.15.132:3300/0 conn(0x7f6eec0cb6e0 0x7f6eec0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2818218925 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143850 0x7f6eec0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2818218925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143470 0x7f6eec06fb10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 -- 172.21.15.88:0/2818218925 >> 172.21.15.88:0/2818218925 conn(0x7f6eec0d0b50 msgr2=0x7f6eec0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 -- 172.21.15.88:0/2818218925 shutdown_connections 2024-04-26T20:00:55.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.990+0000 7f6ef1600640 1 -- 172.21.15.88:0/2818218925 wait complete. 2024-04-26T20:00:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.992+0000 7f6ef1600640 1 Processor -- start 2024-04-26T20:00:55.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.992+0000 7f6ef1600640 1 -- start start 2024-04-26T20:00:55.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6eec0cb6e0 0x7f6eec15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143470 0x7f6eec0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec160c20 con 0x7f6eec143850 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec070180 con 0x7f6eec0cb6e0 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6eec160da0 con 0x7f6eec143470 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53994/0 (socket says 172.21.15.88:53994) 2024-04-26T20:00:55.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef0c00640 1 -- 172.21.15.88:0/2758897931 learned_addr learned my addr 172.21.15.88:0/2758897931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6eec0cb6e0 0x7f6eec15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ee3e00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143470 0x7f6eec0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef0c00640 1 -- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143470 msgr2=0x7f6eec0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.993+0000 7f6ef0c00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143470 0x7f6eec0b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef0c00640 1 -- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6eec0cb6e0 msgr2=0x7f6eec15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef0c00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6eec0cb6e0 0x7f6eec15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef0c00640 1 -- 172.21.15.88:0/2758897931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec0b47a0 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef0c00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400cd70 tx=0x7f6ee400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4003b90 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee4003d30 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec0b4a80 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004240 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.994+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec0b43e0 con 0x7f6eec143850 2024-04-26T20:00:55.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.995+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eec06d1f0 con 0x7f6eec143850 2024-04-26T20:00:55.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.996+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ee40043e0 con 0x7f6eec143850 2024-04-26T20:00:55.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.998+0000 7f6ee8c00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:55.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.998+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5502+0+0 (secure 0 0 0) 0x7f6ee408c570 con 0x7f6eec143850 2024-04-26T20:00:55.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.998+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:55.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:55.998+0000 7f6eebe00640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 0x7f6ec806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6edc034000 tx=0x7f6edc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:56.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.002+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6eec06d1f0 con 0x7f6eec143850 2024-04-26T20:00:56.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.200+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6eec15eba0 con 0x7f6eec143850 2024-04-26T20:00:56.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.773+0000 7f6ee8c00640 1 -- 172.21.15.88:0/2758897931 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f6ee405aa60 con 0x7f6eec143850 2024-04-26T20:00:56.775 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 msgr2=0x7f6ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 0x7f6ec806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6edc034000 tx=0x7f6edc02e000 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 msgr2=0x7f6eec0ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400cd70 tx=0x7f6ee400a660 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 shutdown_connections 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6eec143850 0x7f6eec0ca8f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6ec806c8f0 0x7f6ec806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.777+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6eec143470 0x7f6eec0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.778+0000 7f6ef1600640 1 --2- 172.21.15.88:0/2758897931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6eec0cb6e0 0x7f6eec15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.778+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 >> 172.21.15.88:0/2758897931 conn(0x7f6eec0d0b50 msgr2=0x7f6eec0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.778+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 shutdown_connections 2024-04-26T20:00:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:56.778+0000 7f6ef1600640 1 -- 172.21.15.88:0/2758897931 wait complete. 2024-04-26T20:00:56.806 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:00:56.806 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:00:56.806 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:00:56.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:56.807 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:00:56.807 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:00:56.807 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:00:56.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:56.807 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:00:56.807 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:00:56.808 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:00:56.808 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:56.808 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:00:56.808 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:00:56.808 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:00:56.808 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:00:56.825 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:00:56.825 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:00:56.827 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:00:56.827 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:00:56.830 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:00:56.831 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:00:56.831 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:00:56.831 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:00:56.871 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:00:56.871 DEBUG:teuthology.parallel:result is None 2024-04-26T20:00:56.875 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:00:56.875 DEBUG:teuthology.parallel:result is None 2024-04-26T20:00:56.879 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:00:56.879 DEBUG:teuthology.parallel:result is None 2024-04-26T20:00:56.884 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:00:56.885 DEBUG:teuthology.parallel:result is None 2024-04-26T20:00:56.885 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:00:57.023 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:00:57.024 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-04-26T20:00:57.031 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-26T20:00:57.031 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-26T20:00:57.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.065+0000 7ff2a3e00640 1 Processor -- start 2024-04-26T20:00:57.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.065+0000 7ff2a3e00640 1 -- start start 2024-04-26T20:00:57.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.066+0000 7ff2a3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff2a415f1b0 0x7ff2a40d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.066+0000 7ff2a3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff2a4163650 0x7ff2a40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.066+0000 7ff2a3e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a40d3f50 con 0x7ff2a4162b00 2024-04-26T20:00:57.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.066+0000 7ff2a3e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a40d36b0 con 0x7ff2a415f1b0 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.066+0000 7ff2a3e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a40d3990 con 0x7ff2a4163650 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff2a415f1b0 0x7ff2a40d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff2a4162b00 0x7ff2a4162f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36124/0 (socket says 172.21.15.88:36124) 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2000640 1 -- 172.21.15.88:0/2866672245 learned_addr learned my addr 172.21.15.88:0/2866672245 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a3400640 1 --2- 172.21.15.88:0/2866672245 >> v2:172.21.15.132:3300/0 conn(0x7ff2a4163650 0x7ff2a40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2a00640 1 -- 172.21.15.88:0/2866672245 >> v2:172.21.15.132:3300/0 conn(0x7ff2a4163650 msgr2=0x7ff2a40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:57.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2a00640 1 --2- 172.21.15.88:0/2866672245 >> v2:172.21.15.132:3300/0 conn(0x7ff2a4163650 0x7ff2a40b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2a00640 1 -- 172.21.15.88:0/2866672245 >> v1:172.21.15.88:6789/0 conn(0x7ff2a4162b00 legacy=0x7ff2a4162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.067+0000 7ff2a2a00640 1 -- 172.21.15.88:0/2866672245 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a40d3c70 con 0x7ff2a415f1b0 2024-04-26T20:00:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.068+0000 7ff2a2a00640 1 --2- 172.21.15.88:0/2866672245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a415f1b0 0x7ff2a40d5990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2a415f590 tx=0x7ff29402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9699ec484fe843 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.068+0000 7ff2a1600640 1 -- 172.21.15.88:0/2866672245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294037070 con 0x7ff2a415f1b0 2024-04-26T20:00:57.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.068+0000 7ff2a1600640 1 -- 172.21.15.88:0/2866672245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294032440 con 0x7ff2a415f1b0 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.068+0000 7ff2a1600640 1 -- 172.21.15.88:0/2866672245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294032810 con 0x7ff2a415f1b0 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 -- 172.21.15.88:0/2866672245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a415f1b0 msgr2=0x7ff2a40d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/2866672245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a415f1b0 0x7ff2a40d5990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff2a415f590 tx=0x7ff29402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 -- 172.21.15.88:0/2866672245 shutdown_connections 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/2866672245 >> v2:172.21.15.132:3300/0 conn(0x7ff2a4163650 0x7ff2a40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/2866672245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a415f1b0 0x7ff2a40d5990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 -- 172.21.15.88:0/2866672245 >> 172.21.15.88:0/2866672245 conn(0x7ff2a409e230 msgr2=0x7ff2a40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 -- 172.21.15.88:0/2866672245 shutdown_connections 2024-04-26T20:00:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.069+0000 7ff2a3e00640 1 -- 172.21.15.88:0/2866672245 wait complete. 2024-04-26T20:00:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.071+0000 7ff2a3e00640 1 Processor -- start 2024-04-26T20:00:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.071+0000 7ff2a3e00640 1 -- start start 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a415f1b0 0x7ff2a415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 0x7ff2a40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40b87f0 con 0x7ff2a4163650 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a40d33a0 con 0x7ff2a4162b00 2024-04-26T20:00:57.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a40b8970 con 0x7ff2a415f1b0 2024-04-26T20:00:57.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54044/0 (socket says 172.21.15.88:54044) 2024-04-26T20:00:57.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.072+0000 7ff2a3400640 1 -- 172.21.15.88:0/474806660 learned_addr learned my addr 172.21.15.88:0/474806660 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:57.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a2a00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a415f1b0 0x7ff2a415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a2000640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 0x7ff2a40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 -- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a415f1b0 msgr2=0x7ff2a415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a415f1b0 0x7ff2a415e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 -- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 msgr2=0x7ff2a40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 0x7ff2a40ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 -- 172.21.15.88:0/474806660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a40c9630 con 0x7ff2a4163650 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a2000640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 0x7ff2a40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:57.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3400640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00efc0 tx=0x7ff29c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c018020 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff29c014480 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c009e60 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.073+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a40c9910 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.074+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a40c9ed0 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.075+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a40693c0 con 0x7ff2a4163650 2024-04-26T20:00:57.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.076+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff29c005d10 con 0x7ff2a4163650 2024-04-26T20:00:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.077+0000 7ff29aa00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 0x7ff28006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.077+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7ff29c08c480 con 0x7ff2a4163650 2024-04-26T20:00:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.077+0000 7ff2a2a00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 0x7ff28006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.078+0000 7ff2a2a00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 0x7ff28006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff294034000 tx=0x7ff29402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:57.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.082+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff29c022030 con 0x7ff2a4163650 2024-04-26T20:00:57.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.293+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff2a408d090 con 0x7ff2a4163650 2024-04-26T20:00:57.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.795+0000 7ff29aa00640 1 -- 172.21.15.88:0/474806660 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7ff29c010040 con 0x7ff2a4163650 2024-04-26T20:00:57.796 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-04-26T20:00:57.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.799+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 msgr2=0x7ff28006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.799+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 0x7ff28006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff294034000 tx=0x7ff29402e000 comp rx=0 tx=0).stop 2024-04-26T20:00:57.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.799+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 msgr2=0x7ff2a41f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:57.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.799+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00efc0 tx=0x7ff29c00c490 comp rx=0 tx=0).stop 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 shutdown_connections 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a4163650 0x7ff2a41f9580 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a4162b00 0x7ff2a40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff28006c920 0x7ff28006ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 --2- 172.21.15.88:0/474806660 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a415f1b0 0x7ff2a415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 >> 172.21.15.88:0/474806660 conn(0x7ff2a409e230 msgr2=0x7ff2a40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 shutdown_connections 2024-04-26T20:00:57.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:57.800+0000 7ff2a3e00640 1 -- 172.21.15.88:0/474806660 wait complete. 2024-04-26T20:00:57.825 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:00:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.006+0000 7ff2c9800640 1 Processor -- start 2024-04-26T20:00:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.006+0000 7ff2c9800640 1 -- start start 2024-04-26T20:00:58.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.007+0000 7ff2c9800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff2c406ce80 0x7ff2c40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c9800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c40af9a0 con 0x7ff2c40cd650 2024-04-26T20:00:58.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c9800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c40afc80 con 0x7ff2c415cdf0 2024-04-26T20:00:58.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c9800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c40ae2c0 con 0x7ff2c406ce80 2024-04-26T20:00:58.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c3400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff2c40cd650 0x7ff2c40b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36140/0 (socket says 172.21.15.88:36140) 2024-04-26T20:00:58.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c3400640 1 -- 172.21.15.88:0/874083383 learned_addr learned my addr 172.21.15.88:0/874083383 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c2000640 1 --2- 172.21.15.88:0/874083383 >> v2:172.21.15.132:3300/0 conn(0x7ff2c406ce80 0x7ff2c40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081939009 0 0) 0x7ff2c40af9a0 con 0x7ff2c40cd650 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.008+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2a8003520 con 0x7ff2c40cd650 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 147926060 0 0) 0x7ff2a8003520 con 0x7ff2c40cd650 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 >> v2:172.21.15.132:3300/0 conn(0x7ff2c406ce80 msgr2=0x7ff2c40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 --2- 172.21.15.88:0/874083383 >> v2:172.21.15.132:3300/0 conn(0x7ff2c406ce80 0x7ff2c40add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 >> v1:172.21.15.132:6789/0 conn(0x7ff2c415cdf0 legacy=0x7ff2c406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c40aff60 con 0x7ff2c40cd650 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7ff2bc0019f0 con 0x7ff2c40cd650 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 >> v1:172.21.15.88:6789/0 conn(0x7ff2c40cd650 legacy=0x7ff2c40b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a8004800 0x7ff2a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a80051c0 0x7ff2a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a800c030 con 0x7ff2a80039b0 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a800c1e0 con 0x7ff2a8004800 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a800c3e0 con 0x7ff2a80051c0 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.009+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2000640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a8004800 0x7ff2a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2a00640 1 -- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a80051c0 msgr2=0x7ff2a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2a80051c0 0x7ff2a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2a00640 1 -- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a8004800 msgr2=0x7ff2a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a8004800 0x7ff2a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2a00640 1 -- 172.21.15.88:0/874083383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a8003520 con 0x7ff2a80039b0 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c2000640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2a8004800 0x7ff2a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:00:58.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.010+0000 7ff2c3400640 1 -- 172.21.15.88:0/874083383 reap_dead start 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.011+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2a8003dd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001750 tx=0x7ff2a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae8fcf240cafbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.011+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4001370 con 0x7ff2a80039b0 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.011+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b40052d0 con 0x7ff2a80039b0 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.011+0000 7ff2c1600640 1 -- 172.21.15.88:0/874083383 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b402abe0 con 0x7ff2a80039b0 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 -- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 msgr2=0x7ff2a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2a8003dd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001750 tx=0x7ff2a8005830 comp rx=0 tx=0).stop 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 -- 172.21.15.88:0/874083383 shutdown_connections 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 --2- 172.21.15.88:0/874083383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2a8003dd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 -- 172.21.15.88:0/874083383 >> 172.21.15.88:0/874083383 conn(0x7ff2c40b2d50 msgr2=0x7ff2c40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 -- 172.21.15.88:0/874083383 shutdown_connections 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.012+0000 7ff2c9800640 1 -- 172.21.15.88:0/874083383 wait complete. 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.013+0000 7ff2c9800640 1 Processor -- start 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 -- start start 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2c406f260 0x7ff2c41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2c40cae80 0x7ff2c40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c415d1d0 con 0x7ff2a80039b0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c40b05b0 con 0x7ff2c406f260 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.014+0000 7ff2c9800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c415d350 con 0x7ff2c40cae80 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54068/0 (socket says 172.21.15.88:54068) 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 -- 172.21.15.88:0/3181355962 learned_addr learned my addr 172.21.15.88:0/3181355962 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 -- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2c40cae80 msgr2=0x7ff2c40cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2c40cae80 0x7ff2c40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2000640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2c406f260 0x7ff2c41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 -- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2c406f260 msgr2=0x7ff2c41f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2c406f260 0x7ff2c41f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.015+0000 7ff2c2a00640 1 -- 172.21.15.88:0/3181355962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c41f5a20 con 0x7ff2a80039b0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2c2a00640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001870 tx=0x7ff2b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b403f020 con 0x7ff2a80039b0 2024-04-26T20:00:58.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b4005480 con 0x7ff2a80039b0 2024-04-26T20:00:58.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c41f5dc0 con 0x7ff2a80039b0 2024-04-26T20:00:58.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4007a30 con 0x7ff2a80039b0 2024-04-26T20:00:58.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.016+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c41f6380 con 0x7ff2a80039b0 2024-04-26T20:00:58.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.017+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c4069540 con 0x7ff2a80039b0 2024-04-26T20:00:58.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.018+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2b4042020 con 0x7ff2a80039b0 2024-04-26T20:00:58.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.019+0000 7ff2baa00640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 0x7ff2a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.020+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7ff2b40ab3b0 con 0x7ff2a80039b0 2024-04-26T20:00:58.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.020+0000 7ff2c2000640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 0x7ff2a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.020+0000 7ff2c2000640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 0x7ff2a406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2c41f7fb0 tx=0x7ff2b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.024+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff2b4049030 con 0x7ff2a80039b0 2024-04-26T20:00:58.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.302+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff2c40b57b0 con 0x7ff2a80039b0 2024-04-26T20:00:58.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.303+0000 7ff2baa00640 1 -- 172.21.15.88:0/3181355962 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) ==== 80+0+6645 (secure 0 0 0) 0x7ff2c41f5a20 con 0x7ff2a80039b0 2024-04-26T20:00:58.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:00:58.304 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:00:58.304 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-04-26T20:00:58.304 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0jr1OBAAQRkT9BgmjXURkkrr7MAHWQ==", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmpzxOGBAADbar8X1eXjXWjfB60egiBQ==", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmCnKlCxAAo0jGLXT6bRL3qpWufJJ5Sw==", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.306 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmRfxVJhAAGgPI5k+fa3OuMZs2kt0msg==", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB3ByxmU+QbKBAAKGuevMSCLF8SYrdlt/62YQ==", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.307 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ByxmJeeNMxAA64NhSfVOpwUK5gJ6cQVDvQ==", 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.308 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB+Byxm0QbGAhAA6ufvhdQMCNOPOhioCEovNg==", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCBByxmpToFDxAAAuw5Ew77CrBtkai6YP0AlA==", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.309 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCHByxmou+EHBAAkwkBnJbDkuczBKTj/szbYA==", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCKByxmNUjQNxAAz8gNq0pCsDTYh5Fo7xzwbg==", 2024-04-26T20:00:58.310 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCOByxmDaIOFxAAoLuo0jYUxlFBf/2VpwDkPw==", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.311 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCRByxmaXFiIBAAb4ALfWrAFga124RrQFb4Jw==", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw==", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:00:58.312 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0ByxmbJEcCRAA2X7/8G8AR+PRlmM/59Ebqg==", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmEZsPOxAAJZIw3+ol9NxsWyB851hSRg==", 2024-04-26T20:00:58.313 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0Byxm+tkaEhAAXoURy4RGA/OZsj7sUhk8ZQ==", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:00:58.314 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBxByxmqnYUOBAAkUPOrzy4lbEq83+B/FNZlw==", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mds", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxmUUoxMRAAyF5qxF6PT00QI4mpJZAk7A==", 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.315 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxmk1UxMRAAtg3w+ohpyhZ5t+2JH3+kJA==", 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-osd", 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxm1F4xMRAAcz6f0pWoaPSDZlclBCS/ug==", 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T20:00:58.316 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxmGWoxMRAAh9/o1UvsXECoGyMwRincZQ==", 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxmP3MxMRAAjDlKYqE5T+FNi9csM0AnkQ==", 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZByxmTHwxMRAAAR10HB4madawxKxfZXgA4w==", 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0kFbLxAApmvQFSMZqcrF7d0MGYbgQw==", 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.318 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0PeZJhAAmWcbnXTpRhnfZRdvKzbTTg==", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:00:58.319 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-26T20:00:58.320 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:00:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.306+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 msgr2=0x7ff2a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.306+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 0x7ff2a406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff2c41f7fb0 tx=0x7ff2b0008040 comp rx=0 tx=0).stop 2024-04-26T20:00:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.306+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 msgr2=0x7ff2c4159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.306+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4001870 tx=0x7ff2b4001120 comp rx=0 tx=0).stop 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 shutdown_connections 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff2a406c920 0x7ff2a406ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff2c40cae80 0x7ff2c40cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2c406f260 0x7ff2c41f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 --2- 172.21.15.88:0/3181355962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a80039b0 0x7ff2c4159a00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 >> 172.21.15.88:0/3181355962 conn(0x7ff2c40b2d50 msgr2=0x7ff2c40b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 shutdown_connections 2024-04-26T20:00:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.307+0000 7ff2c9800640 1 -- 172.21.15.88:0/3181355962 wait complete. 2024-04-26T20:00:58.326 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T20:00:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.523+0000 7f32e1000640 1 Processor -- start 2024-04-26T20:00:58.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.524+0000 7f32e1000640 1 -- start start 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32e1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32dc06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32e1000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc073f40 con 0x7f32dc15cb70 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32e1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32dc06d090 con 0x7f32dc15cfd0 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32e1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc0af460 con 0x7f32dc06c770 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32d3e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f32dc15cb70 0x7f32dc073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36150/0 (socket says 172.21.15.88:36150) 2024-04-26T20:00:58.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.525+0000 7f32d3e00640 1 -- 172.21.15.88:0/3973326906 learned_addr learned my addr 172.21.15.88:0/3973326906 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32dc06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352169995 0 0) 0x7f32dc073f40 con 0x7f32dc15cb70 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c0005730 con 0x7f32dc15cb70 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 172500553 0 0) 0x7f32c0005730 con 0x7f32dc15cb70 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 msgr2=0x7f32dc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32dc06cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 >> v1:172.21.15.132:6789/0 conn(0x7f32dc15cfd0 legacy=0x7f32dc073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc0ae850 con 0x7f32dc15cb70 2024-04-26T20:00:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f32c4001670 con 0x7f32dc15cb70 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32dc06cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 >> v1:172.21.15.88:6789/0 conn(0x7f32dc15cb70 legacy=0x7f32dc073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32c0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c00061e0 0x7f32c0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.526+0000 7f32d9400640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0006f50 0x7f32c00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0af460 con 0x7f32c00061e0 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f32c000dd30 con 0x7f32c0006f50 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f32c000df30 con 0x7f32dc06c770 2024-04-26T20:00:58.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c00061e0 0x7f32c0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d9e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32c0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 -- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 msgr2=0x7f32c0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32dc06c770 0x7f32c0005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 -- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0006f50 msgr2=0x7f32c00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c0006f50 0x7f32c00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.527+0000 7f32d3e00640 1 -- 172.21.15.88:0/3973326906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c0005730 con 0x7f32c00061e0 2024-04-26T20:00:58.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.528+0000 7f32d3e00640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c00061e0 0x7f32c0006a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c210 tx=0x7f32c0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb59ca52faa149f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.528+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007d30 con 0x7f32c00061e0 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.528+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c4005af0 con 0x7f32c00061e0 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.528+0000 7f32d9400640 1 -- 172.21.15.88:0/3973326906 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007070 con 0x7f32c00061e0 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 -- 172.21.15.88:0/3973326906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c00061e0 msgr2=0x7f32c0006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 --2- 172.21.15.88:0/3973326906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c00061e0 0x7f32c0006a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c210 tx=0x7f32c0007a40 comp rx=0 tx=0).stop 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32da800640 1 -- 172.21.15.88:0/3973326906 reap_dead start 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 -- 172.21.15.88:0/3973326906 shutdown_connections 2024-04-26T20:00:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 -- 172.21.15.88:0/3973326906 >> 172.21.15.88:0/3973326906 conn(0x7f32dc0b2d50 msgr2=0x7f32dc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 -- 172.21.15.88:0/3973326906 shutdown_connections 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.529+0000 7f32e1000640 1 -- 172.21.15.88:0/3973326906 wait complete. 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.531+0000 7f32e1000640 1 Processor -- start 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.531+0000 7f32e1000640 1 -- start start 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c00061e0 0x7f32dc14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32dc0733a0 0x7f32dc15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0af0f0 con 0x7f32dc152130 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc0aee10 con 0x7f32dc0733a0 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32e1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f32dc15c600 con 0x7f32c00061e0 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32d3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54084/0 (socket says 172.21.15.88:54084) 2024-04-26T20:00:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.532+0000 7f32d3e00640 1 -- 172.21.15.88:0/2262488314 learned_addr learned my addr 172.21.15.88:0/2262488314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d9e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c00061e0 0x7f32dc14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 -- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c00061e0 msgr2=0x7f32dc14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c00061e0 0x7f32dc14faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 -- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32dc0733a0 msgr2=0x7f32dc15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32dc0733a0 0x7f32dc15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 -- 172.21.15.88:0/2262488314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc073cf0 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d3e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c270 tx=0x7f32c4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4041020 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c4001470 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc0cd7f0 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007270 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.533+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc0cddb0 con 0x7f32dc152130 2024-04-26T20:00:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.535+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc1608a0 con 0x7f32dc152130 2024-04-26T20:00:58.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.535+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32c4044020 con 0x7f32dc152130 2024-04-26T20:00:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.536+0000 7f32d2000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 0x7f32b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.536+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f32c40ab9a0 con 0x7f32dc152130 2024-04-26T20:00:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.537+0000 7f32d9e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 0x7f32b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.537+0000 7f32d9e00640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 0x7f32b806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.541+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32dc1608a0 con 0x7f32dc152130 2024-04-26T20:00:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.819+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f32dc069540 con 0x7f32dc152130 2024-04-26T20:00:58.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.823+0000 7f32d2000640 1 -- 172.21.15.88:0/2262488314 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f32c407a110 con 0x7f32dc152130 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 msgr2=0x7f32b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 0x7f32b806ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc008040 comp rx=0 tx=0).stop 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 msgr2=0x7f32dc14f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c270 tx=0x7f32c4001a40 comp rx=0 tx=0).stop 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 shutdown_connections 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32dc0733a0 0x7f32dc15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32dc152130 0x7f32dc14f2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f32b806c920 0x7f32b806ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 --2- 172.21.15.88:0/2262488314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f32c00061e0 0x7f32dc14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 >> 172.21.15.88:0/2262488314 conn(0x7f32dc0b2d50 msgr2=0x7f32dc15f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 shutdown_connections 2024-04-26T20:00:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:58.826+0000 7f32e1000640 1 -- 172.21.15.88:0/2262488314 wait complete. 2024-04-26T20:00:58.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T20:00:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.045+0000 7f6000a00640 1 Processor -- start 2024-04-26T20:00:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.046+0000 7f6000a00640 1 -- start start 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.047+0000 7f6000a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5ffc0af670 0x7f5ffc15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.047+0000 7f6000a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc0afb50 con 0x7f5ffc15d720 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f6000a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ffc0afe30 con 0x7f5ffc0cf640 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f6000a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f5ffc0b0110 con 0x7f5ffc0af670 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ff3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5ffc0af670 0x7f5ffc15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ffbe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5ffc15d720 0x7f5ffc0d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50092/0 (socket says 172.21.15.88:50092) 2024-04-26T20:00:59.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ffbe00640 1 -- 172.21.15.88:0/1789149000 learned_addr learned my addr 172.21.15.88:0/1789149000 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:00:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ff3e00640 1 -- 172.21.15.88:0/1789149000 >> v1:172.21.15.132:6789/0 conn(0x7f5ffc0cf640 legacy=0x7f5ffc0b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ff3e00640 1 -- 172.21.15.88:0/1789149000 >> v1:172.21.15.88:6789/0 conn(0x7f5ffc15d720 legacy=0x7f5ffc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ffaa00640 1 -- 172.21.15.88:0/1789149000 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465452372 0 0) 0x7f5ffc0afb50 con 0x7f5ffc15d720 2024-04-26T20:00:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.048+0000 7f5ff3e00640 1 -- 172.21.15.88:0/1789149000 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc0b03f0 con 0x7f5ffc0af670 2024-04-26T20:00:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.049+0000 7f5ff3e00640 1 --2- 172.21.15.88:0/1789149000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc0af670 0x7f5ffc15d1e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40097a0 tx=0x7f5fe402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baa9af46c9c9844 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.050+0000 7f5ffaa00640 1 -- 172.21.15.88:0/1789149000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4037070 con 0x7f5ffc0af670 2024-04-26T20:00:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.050+0000 7f5ffaa00640 1 -- 172.21.15.88:0/1789149000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe4032440 con 0x7f5ffc0af670 2024-04-26T20:00:59.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.050+0000 7f5ffaa00640 1 -- 172.21.15.88:0/1789149000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4032810 con 0x7f5ffc0af670 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.050+0000 7f6000a00640 1 -- 172.21.15.88:0/1789149000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc0af670 msgr2=0x7f5ffc15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.050+0000 7f6000a00640 1 --2- 172.21.15.88:0/1789149000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc0af670 0x7f5ffc15d1e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40097a0 tx=0x7f5fe402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.051+0000 7f6000a00640 1 -- 172.21.15.88:0/1789149000 shutdown_connections 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.051+0000 7f6000a00640 1 --2- 172.21.15.88:0/1789149000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc0af670 0x7f5ffc15d1e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.051+0000 7f6000a00640 1 -- 172.21.15.88:0/1789149000 >> 172.21.15.88:0/1789149000 conn(0x7f5ffc06c560 msgr2=0x7f5ffc06c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:59.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.051+0000 7f6000a00640 1 -- 172.21.15.88:0/1789149000 shutdown_connections 2024-04-26T20:00:59.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.051+0000 7f6000a00640 1 -- 172.21.15.88:0/1789149000 wait complete. 2024-04-26T20:00:59.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.052+0000 7f6000a00640 1 Processor -- start 2024-04-26T20:00:59.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.053+0000 7f6000a00640 1 -- start start 2024-04-26T20:00:59.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.053+0000 7f6000a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ffc0af670 0x7f5ffc072a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.053+0000 7f6000a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ffc0cf640 0x7f5ffc1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f6000a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f6000a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0d1620 con 0x7f5ffc15d720 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f6000a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0d14a0 con 0x7f5ffc0cf640 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f6000a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc160d50 con 0x7f5ffc0af670 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52784/0 (socket says 172.21.15.88:52784) 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 -- 172.21.15.88:0/2862822351 learned_addr learned my addr 172.21.15.88:0/2862822351 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ff3e00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ffc0cf640 0x7f5ffc1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffb400640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ffc0af670 0x7f5ffc072a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 -- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ffc0af670 msgr2=0x7f5ffc072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ffc0af670 0x7f5ffc072a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 -- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ffc0cf640 msgr2=0x7f5ffc1f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ffc0cf640 0x7f5ffc1f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.054+0000 7f5ffbe00640 1 -- 172.21.15.88:0/2862822351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc1f9120 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f5ffbe00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c700 tx=0x7f5ff400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4018070 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff400ce10 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400a860 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc1f50c0 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.055+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc072f50 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.056+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc0af250 con 0x7f5ffc15d720 2024-04-26T20:00:59.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.057+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ff400aa00 con 0x7f5ffc15d720 2024-04-26T20:00:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.058+0000 7f5ff3400640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.058+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5ff408ad60 con 0x7f5ffc15d720 2024-04-26T20:00:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.058+0000 7f5ffb400640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.059+0000 7f5ffb400640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 0x7f5fd806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fec003d90 tx=0x7f5fec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.062+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ffc0af250 con 0x7f5ffc15d720 2024-04-26T20:00:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.332+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f5ffc1f4eb0 con 0x7f5ffc15d720 2024-04-26T20:00:59.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.337+0000 7f5ff3400640 1 -- 172.21.15.88:0/2862822351 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f5ff4059440 con 0x7f5ffc15d720 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.339+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 msgr2=0x7f5fd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 0x7f5fd806ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fec003d90 tx=0x7f5fec009040 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 msgr2=0x7f5ffc1f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c700 tx=0x7f5ff400cbc0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 shutdown_connections 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ffc15d720 0x7f5ffc1f89e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ffc0cf640 0x7f5ffc1f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5fd806c920 0x7f5fd806ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 --2- 172.21.15.88:0/2862822351 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5ffc0af670 0x7f5ffc072a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 >> 172.21.15.88:0/2862822351 conn(0x7f5ffc06c560 msgr2=0x7f5ffc073930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:59.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 shutdown_connections 2024-04-26T20:00:59.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.340+0000 7f6000a00640 1 -- 172.21.15.88:0/2862822351 wait complete. 2024-04-26T20:00:59.358 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T20:00:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.545+0000 7fed27200640 1 Processor -- start 2024-04-26T20:00:59.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.546+0000 7fed27200640 1 -- start start 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.547+0000 7fed27200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed280cf640 0x7fed280b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.547+0000 7fed27200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fed280af670 0x7fed2815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.547+0000 7fed27200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed280afb50 con 0x7fed2815d720 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.547+0000 7fed27200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fed280afe30 con 0x7fed280af670 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.547+0000 7fed27200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fed280b0110 con 0x7fed280cf640 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fed280af670 0x7fed2815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fed280af670 0x7fed2815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52798/0 (socket says 172.21.15.88:52798) 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 -- 172.21.15.88:0/3082988543 learned_addr learned my addr 172.21.15.88:0/3082988543 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed25e00640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed280cf640 0x7fed280b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 -- 172.21.15.88:0/3082988543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed280cf640 msgr2=0x7fed280b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:59.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed280cf640 0x7fed280b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 -- 172.21.15.88:0/3082988543 >> v1:172.21.15.132:6789/0 conn(0x7fed2815d720 legacy=0x7fed280d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:00:59.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.548+0000 7fed1fe00640 1 -- 172.21.15.88:0/3082988543 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed280b03f0 con 0x7fed280af670 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.549+0000 7fed1fe00640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed2815d1e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca1021f4aa231a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed25400640 1 -- 172.21.15.88:0/3082988543 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10037070 con 0x7fed280af670 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed25400640 1 -- 172.21.15.88:0/3082988543 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10032440 con 0x7fed280af670 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed25400640 1 -- 172.21.15.88:0/3082988543 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed10032810 con 0x7fed280af670 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 -- 172.21.15.88:0/3082988543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 msgr2=0x7fed2815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed2815d1e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002a210 comp rx=0 tx=0).stop 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 -- 172.21.15.88:0/3082988543 shutdown_connections 2024-04-26T20:00:59.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed2815d1e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 --2- 172.21.15.88:0/3082988543 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed280cf640 0x7fed280b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.550+0000 7fed27200640 1 -- 172.21.15.88:0/3082988543 >> 172.21.15.88:0/3082988543 conn(0x7fed2806c560 msgr2=0x7fed2806c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:59.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.551+0000 7fed27200640 1 -- 172.21.15.88:0/3082988543 shutdown_connections 2024-04-26T20:00:59.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.551+0000 7fed27200640 1 -- 172.21.15.88:0/3082988543 wait complete. 2024-04-26T20:00:59.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.552+0000 7fed27200640 1 Processor -- start 2024-04-26T20:00:59.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.552+0000 7fed27200640 1 -- start start 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.553+0000 7fed27200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.553+0000 7fed27200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed280cf640 0x7fed281f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.553+0000 7fed27200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed2815d720 0x7fed28156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.553+0000 7fed27200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed280b5e60 con 0x7fed280af670 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.553+0000 7fed27200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fed280d16f0 con 0x7fed280cf640 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed27200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fed280b5fe0 con 0x7fed2815d720 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52814/0 (socket says 172.21.15.88:52814) 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 -- 172.21.15.88:0/3964040948 learned_addr learned my addr 172.21.15.88:0/3964040948 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:00:59.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 -- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed2815d720 msgr2=0x7fed28156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed2815d720 0x7fed28156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 -- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed280cf640 msgr2=0x7fed281f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed280cf640 0x7fed281f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.554+0000 7fed25e00640 1 -- 172.21.15.88:0/3964040948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed281f70a0 con 0x7fed280af670 2024-04-26T20:00:59.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed25e00640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c7f0 tx=0x7fed1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18015070 con 0x7fed280af670 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed18004240 con 0x7fed280af670 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18007620 con 0x7fed280af670 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed280b07d0 con 0x7fed280af670 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.555+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed280b0d90 con 0x7fed280af670 2024-04-26T20:00:59.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.556+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed280693c0 con 0x7fed280af670 2024-04-26T20:00:59.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.557+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed180079d0 con 0x7fed280af670 2024-04-26T20:00:59.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.558+0000 7fed1e000640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:00:59.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.558+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fed1808bbe0 con 0x7fed280af670 2024-04-26T20:00:59.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.558+0000 7fed1fe00640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:00:59.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.559+0000 7fed1fe00640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 0x7fed0406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:00:59.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.562+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed1805a390 con 0x7fed280af670 2024-04-26T20:00:59.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.828+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fed2806f680 con 0x7fed280af670 2024-04-26T20:00:59.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.835+0000 7fed1e000640 1 -- 172.21.15.88:0/3964040948 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fed18059c90 con 0x7fed280af670 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 msgr2=0x7fed0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 0x7fed0406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002c000 comp rx=0 tx=0).stop 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 msgr2=0x7fed28153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c7f0 tx=0x7fed1800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 shutdown_connections 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fed0406c8f0 0x7fed0406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fed2815d720 0x7fed28156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed280cf640 0x7fed281f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 --2- 172.21.15.88:0/3964040948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed280af670 0x7fed28153b80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:00:59.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 >> 172.21.15.88:0/3964040948 conn(0x7fed2806c560 msgr2=0x7fed2809da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:00:59.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.837+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 shutdown_connections 2024-04-26T20:00:59.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:00:59.838+0000 7fed27200640 1 -- 172.21.15.88:0/3964040948 wait complete. 2024-04-26T20:00:59.856 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T20:01:00.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.048+0000 7f1cc3800640 1 Processor -- start 2024-04-26T20:01:00.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.048+0000 7f1cc3800640 1 -- start start 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 0x7f1cbc0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 0x7f1cbc06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0ae2c0 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0b05b0 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc3800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0af9a0 con 0x7f1cbc0cd650 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc1800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc1800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52816/0 (socket says 172.21.15.88:52816) 2024-04-26T20:01:00.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.050+0000 7f1cc1800640 1 -- 172.21.15.88:0/658601723 learned_addr learned my addr 172.21.15.88:0/658601723 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc0e00640 1 --2- 172.21.15.88:0/658601723 >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 0x7f1cbc0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc1800640 1 -- 172.21.15.88:0/658601723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 msgr2=0x7f1cbc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cb3e00640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 0x7f1cbc06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc1800640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 0x7f1cbc06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc1800640 1 -- 172.21.15.88:0/658601723 >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 msgr2=0x7f1cbc0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc1800640 1 --2- 172.21.15.88:0/658601723 >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 0x7f1cbc0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc1800640 1 -- 172.21.15.88:0/658601723 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0afc80 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cc0e00640 1 --2- 172.21.15.88:0/658601723 >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 0x7f1cbc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.051+0000 7f1cb3e00640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 0x7f1cbc06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.052+0000 7f1cc1800640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0b3950 tx=0x7f1cb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e2a774a737665c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.052+0000 7f1cbbe00640 1 -- 172.21.15.88:0/658601723 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4036070 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.052+0000 7f1cbbe00640 1 -- 172.21.15.88:0/658601723 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb4031600 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.052+0000 7f1cbbe00640 1 -- 172.21.15.88:0/658601723 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40319d0 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 -- 172.21.15.88:0/658601723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc0b4f50 msgr2=0x7f1cbc06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0b3950 tx=0x7f1cb402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 -- 172.21.15.88:0/658601723 shutdown_connections 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc0b4f50 0x7f1cbc06ccb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 --2- 172.21.15.88:0/658601723 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0cd650 0x7f1cbc06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 --2- 172.21.15.88:0/658601723 >> v2:172.21.15.132:3300/0 conn(0x7f1cbc15cdf0 0x7f1cbc0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 -- 172.21.15.88:0/658601723 >> 172.21.15.88:0/658601723 conn(0x7f1cbc0b2d50 msgr2=0x7f1cbc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 -- 172.21.15.88:0/658601723 shutdown_connections 2024-04-26T20:01:00.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.053+0000 7f1cc3800640 1 -- 172.21.15.88:0/658601723 wait complete. 2024-04-26T20:01:00.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.055+0000 7f1cc3800640 1 Processor -- start 2024-04-26T20:01:00.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.055+0000 7f1cc3800640 1 -- start start 2024-04-26T20:01:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0b4f50 0x7f1cbc09d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc0cd650 0x7f1cbc0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc15d1d0 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0b0240 con 0x7f1cbc0cd650 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc3800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc15d350 con 0x7f1cbc0b4f50 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52826/0 (socket says 172.21.15.88:52826) 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.056+0000 7f1cc1800640 1 -- 172.21.15.88:0/1228332933 learned_addr learned my addr 172.21.15.88:0/1228332933 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc0e00640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0b4f50 0x7f1cbc09d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 -- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0b4f50 msgr2=0x7f1cbc09d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0b4f50 0x7f1cbc09d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 -- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc0cd650 msgr2=0x7f1cbc0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc0cd650 0x7f1cbc0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 -- 172.21.15.88:0/1228332933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc09dc10 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc1800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1cb402ac90 tx=0x7f1cb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403f020 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb4037040 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4036070 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc09c630 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.057+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc09cbf0 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.058+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc069540 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.059+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cb4042020 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.060+0000 7f1cb9600640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.060+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f1cb40abd00 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.061+0000 7f1cc0e00640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.061+0000 7f1cc0e00640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 0x7f1c9806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06fc40 tx=0x7f1ca4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.064+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cb4049030 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.323+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1cbc076760 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.328+0000 7f1cb9600640 1 -- 172.21.15.88:0/1228332933 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f1cb407a510 con 0x7f1cbc15cdf0 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 msgr2=0x7f1c9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 0x7f1c9806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06fc40 tx=0x7f1ca4008040 comp rx=0 tx=0).stop 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 msgr2=0x7f1cbc09ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1cb402ac90 tx=0x7f1cb4004a40 comp rx=0 tx=0).stop 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 shutdown_connections 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc15cdf0 0x7f1cbc09ba50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc0cd650 0x7f1cbc0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1c9806c8f0 0x7f1c9806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 --2- 172.21.15.88:0/1228332933 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1cbc0b4f50 0x7f1cbc09d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 >> 172.21.15.88:0/1228332933 conn(0x7f1cbc0b2d50 msgr2=0x7f1cbc0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:00.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 shutdown_connections 2024-04-26T20:01:00.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.331+0000 7f1cc3800640 1 -- 172.21.15.88:0/1228332933 wait complete. 2024-04-26T20:01:00.349 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T20:01:00.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.543+0000 7f5e6be00640 1 Processor -- start 2024-04-26T20:01:00.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.544+0000 7f5e6be00640 1 -- start start 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e6be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5e6415cdf0 0x7f5e640add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6406ce80 0x7f5e640b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e6be00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e640af9a0 con 0x7f5e640cd650 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e6be00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f5e640ae2c0 con 0x7f5e6415cdf0 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e6be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5e640b05b0 con 0x7f5e6406ce80 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e68a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f5e640cd650 0x7f5e6406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50098/0 (socket says 172.21.15.88:50098) 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e69400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5e6415cdf0 0x7f5e640add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.545+0000 7f5e68a00640 1 -- 172.21.15.88:0/1221705852 learned_addr learned my addr 172.21.15.88:0/1221705852 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.546+0000 7f5e69e00640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6406ce80 0x7f5e640b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.546+0000 7f5e69400640 1 -- 172.21.15.88:0/1221705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6406ce80 msgr2=0x7f5e640b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.546+0000 7f5e69400640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6406ce80 0x7f5e640b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.546+0000 7f5e69400640 1 -- 172.21.15.88:0/1221705852 >> v1:172.21.15.88:6789/0 conn(0x7f5e640cd650 legacy=0x7f5e6406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.546+0000 7f5e69400640 1 -- 172.21.15.88:0/1221705852 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640afc80 con 0x7f5e6415cdf0 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.547+0000 7f5e69400640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415cdf0 0x7f5e640add80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e580097a0 tx=0x7f5e5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced1f7cd3a82ece4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.547+0000 7f5e63e00640 1 -- 172.21.15.88:0/1221705852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58037070 con 0x7f5e6415cdf0 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.547+0000 7f5e63e00640 1 -- 172.21.15.88:0/1221705852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e58032440 con 0x7f5e6415cdf0 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.547+0000 7f5e63e00640 1 -- 172.21.15.88:0/1221705852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58032810 con 0x7f5e6415cdf0 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 -- 172.21.15.88:0/1221705852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415cdf0 msgr2=0x7f5e640add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415cdf0 0x7f5e640add80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e580097a0 tx=0x7f5e5802a180 comp rx=0 tx=0).stop 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 -- 172.21.15.88:0/1221705852 shutdown_connections 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6406ce80 0x7f5e640b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 --2- 172.21.15.88:0/1221705852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415cdf0 0x7f5e640add80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 -- 172.21.15.88:0/1221705852 >> 172.21.15.88:0/1221705852 conn(0x7f5e640b2d50 msgr2=0x7f5e640b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 -- 172.21.15.88:0/1221705852 shutdown_connections 2024-04-26T20:01:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.548+0000 7f5e6be00640 1 -- 172.21.15.88:0/1221705852 wait complete. 2024-04-26T20:01:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.550+0000 7f5e6be00640 1 Processor -- start 2024-04-26T20:01:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.550+0000 7f5e6be00640 1 -- start start 2024-04-26T20:01:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e640cd650 0x7f5e640cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6415cdf0 0x7f5e64158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6415d1d0 con 0x7f5e6406ce80 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640b0240 con 0x7f5e640cd650 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e6be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6415d350 con 0x7f5e6415cdf0 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52850/0 (socket says 172.21.15.88:52850) 2024-04-26T20:01:00.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 -- 172.21.15.88:0/2946271036 learned_addr learned my addr 172.21.15.88:0/2946271036 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 -- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6415cdf0 msgr2=0x7f5e64158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6415cdf0 0x7f5e64158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 -- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e640cd650 msgr2=0x7f5e640cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e640cd650 0x7f5e640cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.551+0000 7f5e69400640 1 -- 172.21.15.88:0/2946271036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641491c0 con 0x7f5e6406ce80 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e69400640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e58035040 tx=0x7f5e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58037040 con 0x7f5e6406ce80 2024-04-26T20:01:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e58030770 con 0x7f5e6406ce80 2024-04-26T20:01:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58030b80 con 0x7f5e6406ce80 2024-04-26T20:01:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e64149460 con 0x7f5e6406ce80 2024-04-26T20:01:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.552+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e64149a20 con 0x7f5e6406ce80 2024-04-26T20:01:00.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.553+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e64069540 con 0x7f5e6406ce80 2024-04-26T20:01:00.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.554+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e58044020 con 0x7f5e6406ce80 2024-04-26T20:01:00.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.555+0000 7f5e61600640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:00.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.555+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5e580abe10 con 0x7f5e6406ce80 2024-04-26T20:01:00.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.556+0000 7f5e68a00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:00.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.556+0000 7f5e68a00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 0x7f5e4406ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000f60 tx=0x7f5e50007a80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:00.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.560+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e580af030 con 0x7f5e6406ce80 2024-04-26T20:01:00.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.826+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f5e640b5840 con 0x7f5e6406ce80 2024-04-26T20:01:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.839+0000 7f5e61600640 1 -- 172.21.15.88:0/2946271036 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) ==== 97+0+0 (secure 0 0 0) 0x7f5e58032e20 con 0x7f5e6406ce80 2024-04-26T20:01:00.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 msgr2=0x7f5e4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 0x7f5e4406ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000f60 tx=0x7f5e50007a80 comp rx=0 tx=0).stop 2024-04-26T20:01:00.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 msgr2=0x7f5e641f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e58035040 tx=0x7f5e580046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 shutdown_connections 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5e4406c920 0x7f5e4406ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5e6415cdf0 0x7f5e64158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e640cd650 0x7f5e640cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 --2- 172.21.15.88:0/2946271036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6406ce80 0x7f5e641f4dc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.841+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 >> 172.21.15.88:0/2946271036 conn(0x7f5e640b2d50 msgr2=0x7f5e640b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.842+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 shutdown_connections 2024-04-26T20:01:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:00.842+0000 7f5e6be00640 1 -- 172.21.15.88:0/2946271036 wait complete. 2024-04-26T20:01:00.860 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T20:01:01.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.046+0000 7f8ece800640 1 Processor -- start 2024-04-26T20:01:01.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.046+0000 7f8ece800640 1 -- start start 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ece800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ece800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8ec80b5e60 0x7f8ec80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ece800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec80afb50 con 0x7f8ec80af670 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ece800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec80afe30 con 0x7f8ec80b5e60 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ece800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec80b0110 con 0x7f8ec80cf640 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ecde00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8ec80b5e60 0x7f8ec80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ecde00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8ec80b5e60 0x7f8ec80d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52856/0 (socket says 172.21.15.88:52856) 2024-04-26T20:01:01.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.048+0000 7f8ecde00640 1 -- 172.21.15.88:0/1854644590 learned_addr learned my addr 172.21.15.88:0/1854644590 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.049+0000 7f8ecd400640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.049+0000 7f8ecde00640 1 -- 172.21.15.88:0/1854644590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 msgr2=0x7f8ec80b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.049+0000 7f8ecde00640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.049+0000 7f8ecde00640 1 -- 172.21.15.88:0/1854644590 >> v1:172.21.15.132:6789/0 conn(0x7f8ec80af670 legacy=0x7f8ec815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.049+0000 7f8ecde00640 1 -- 172.21.15.88:0/1854644590 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec80b03f0 con 0x7f8ec80b5e60 2024-04-26T20:01:01.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.050+0000 7f8ecde00640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80d0d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009e50 tx=0x7f8ec002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1233e248cfcb186d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ec7e00640 1 -- 172.21.15.88:0/1854644590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0036070 con 0x7f8ec80b5e60 2024-04-26T20:01:01.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ec7e00640 1 -- 172.21.15.88:0/1854644590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec0031600 con 0x7f8ec80b5e60 2024-04-26T20:01:01.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ec7e00640 1 -- 172.21.15.88:0/1854644590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00319d0 con 0x7f8ec80b5e60 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 -- 172.21.15.88:0/1854644590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80b5e60 msgr2=0x7f8ec80d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80d0d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009e50 tx=0x7f8ec002a710 comp rx=0 tx=0).stop 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 -- 172.21.15.88:0/1854644590 shutdown_connections 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80d0d90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 --2- 172.21.15.88:0/1854644590 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.051+0000 7f8ece800640 1 -- 172.21.15.88:0/1854644590 >> 172.21.15.88:0/1854644590 conn(0x7f8ec806c560 msgr2=0x7f8ec806c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:01.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.052+0000 7f8ece800640 1 -- 172.21.15.88:0/1854644590 shutdown_connections 2024-04-26T20:01:01.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.052+0000 7f8ece800640 1 -- 172.21.15.88:0/1854644590 wait complete. 2024-04-26T20:01:01.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.053+0000 7f8ece800640 1 Processor -- start 2024-04-26T20:01:01.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.053+0000 7f8ece800640 1 -- start start 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec80d1450 con 0x7f8ec80af670 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec80d12d0 con 0x7f8ec80b5e60 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ece800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec80d15d0 con 0x7f8ec80cf640 2024-04-26T20:01:01.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ecd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52872/0 (socket says 172.21.15.88:52872) 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.054+0000 7f8ecd400640 1 -- 172.21.15.88:0/1015791251 learned_addr learned my addr 172.21.15.88:0/1015791251 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 -- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 msgr2=0x7f8ec80b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 -- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ec80b5e60 msgr2=0x7f8ec80b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 -- 172.21.15.88:0/1015791251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec809e7e0 con 0x7f8ec80af670 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ecd400640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00d5f0 tx=0x7f8ebc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0125f0 con 0x7f8ec80af670 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ebc019070 con 0x7f8ec80af670 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0129d0 con 0x7f8ec80af670 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec80b1920 con 0x7f8ec80af670 2024-04-26T20:01:01.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.055+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec80b1ee0 con 0x7f8ec80af670 2024-04-26T20:01:01.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.057+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec80693c0 con 0x7f8ec80af670 2024-04-26T20:01:01.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.058+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ebc012b70 con 0x7f8ec80af670 2024-04-26T20:01:01.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.058+0000 7f8ec5600640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.059+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f8ebc08bbc0 con 0x7f8ec80af670 2024-04-26T20:01:01.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.059+0000 7f8ecca00640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.059+0000 7f8ecca00640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 0x7f8ea806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4000f60 tx=0x7f8eb40079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.063+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ebc05a3d0 con 0x7f8ec80af670 2024-04-26T20:01:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.342+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f8ec806f680 con 0x7f8ec80af670 2024-04-26T20:01:01.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.347+0000 7f8ec5600640 1 -- 172.21.15.88:0/1015791251 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f8ebc059cd0 con 0x7f8ec80af670 2024-04-26T20:01:01.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.349+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 msgr2=0x7f8ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.349+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 0x7f8ea806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4000f60 tx=0x7f8eb40079d0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.349+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 msgr2=0x7f8ec81518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.349+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00d5f0 tx=0x7f8ebc00dab0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 shutdown_connections 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8ea806c8f0 0x7f8ea806edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8ec80cf640 0x7f8ec80b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ec80b5e60 0x7f8ec80b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 --2- 172.21.15.88:0/1015791251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ec80af670 0x7f8ec81518e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 >> 172.21.15.88:0/1015791251 conn(0x7f8ec806c560 msgr2=0x7f8ec80b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 shutdown_connections 2024-04-26T20:01:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.350+0000 7f8ece800640 1 -- 172.21.15.88:0/1015791251 wait complete. 2024-04-26T20:01:01.369 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.566+0000 7f58a8a00640 1 Processor -- start 2024-04-26T20:01:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.567+0000 7f58a8a00640 1 -- start start 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a8a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a415f1b0 0x7f58a40d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a8a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f58a4163650 0x7f58a40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a8a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a40d3f50 con 0x7f58a4162b00 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a8a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f58a40d36b0 con 0x7f58a415f1b0 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a8a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f58a40d3990 con 0x7f58a4163650 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a415f1b0 0x7f58a40d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a3400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f58a415f1b0 0x7f58a40d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52884/0 (socket says 172.21.15.88:52884) 2024-04-26T20:01:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.568+0000 7f58a3400640 1 -- 172.21.15.88:0/2805610126 learned_addr learned my addr 172.21.15.88:0/2805610126 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.569+0000 7f58a3400640 1 -- 172.21.15.88:0/2805610126 >> v2:172.21.15.132:3300/0 conn(0x7f58a4163650 msgr2=0x7f58a40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.569+0000 7f58a3400640 1 --2- 172.21.15.88:0/2805610126 >> v2:172.21.15.132:3300/0 conn(0x7f58a4163650 0x7f58a40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.569+0000 7f58a3400640 1 -- 172.21.15.88:0/2805610126 >> v1:172.21.15.132:6789/0 conn(0x7f58a4162b00 legacy=0x7f58a4162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.569+0000 7f58a3400640 1 -- 172.21.15.88:0/2805610126 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a40d3c70 con 0x7f58a415f1b0 2024-04-26T20:01:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.570+0000 7f58a3400640 1 --2- 172.21.15.88:0/2805610126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a415f1b0 0x7f58a40d5990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58a415f590 tx=0x7f589402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d892a19ba3ad33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.570+0000 7f58a2000640 1 -- 172.21.15.88:0/2805610126 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894037070 con 0x7f58a415f1b0 2024-04-26T20:01:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.570+0000 7f58a2000640 1 -- 172.21.15.88:0/2805610126 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5894032440 con 0x7f58a415f1b0 2024-04-26T20:01:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.570+0000 7f58a2000640 1 -- 172.21.15.88:0/2805610126 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894032810 con 0x7f58a415f1b0 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 -- 172.21.15.88:0/2805610126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a415f1b0 msgr2=0x7f58a40d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 --2- 172.21.15.88:0/2805610126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a415f1b0 0x7f58a40d5990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58a415f590 tx=0x7f589402a230 comp rx=0 tx=0).stop 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 -- 172.21.15.88:0/2805610126 shutdown_connections 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 --2- 172.21.15.88:0/2805610126 >> v2:172.21.15.132:3300/0 conn(0x7f58a4163650 0x7f58a40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 --2- 172.21.15.88:0/2805610126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a415f1b0 0x7f58a40d5990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 -- 172.21.15.88:0/2805610126 >> 172.21.15.88:0/2805610126 conn(0x7f58a409e230 msgr2=0x7f58a40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 -- 172.21.15.88:0/2805610126 shutdown_connections 2024-04-26T20:01:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.571+0000 7f58a8a00640 1 -- 172.21.15.88:0/2805610126 wait complete. 2024-04-26T20:01:01.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.573+0000 7f58a8a00640 1 Processor -- start 2024-04-26T20:01:01.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.573+0000 7f58a8a00640 1 -- start start 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415f1b0 0x7f58a415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4162b00 0x7f58a40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58a40b87f0 con 0x7f58a4163650 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f58a40d33a0 con 0x7f58a4162b00 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f58a40b8970 con 0x7f58a415f1b0 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52890/0 (socket says 172.21.15.88:52890) 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 -- 172.21.15.88:0/3647859880 learned_addr learned my addr 172.21.15.88:0/3647859880 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a2a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4162b00 0x7f58a40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3400640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415f1b0 0x7f58a415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 -- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415f1b0 msgr2=0x7f58a415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415f1b0 0x7f58a415d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 -- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4162b00 msgr2=0x7f58a40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4162b00 0x7f58a40cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.574+0000 7f58a3e00640 1 -- 172.21.15.88:0/3647859880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a415e410 con 0x7f58a4163650 2024-04-26T20:01:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f58a3e00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b410 tx=0x7f589c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c015020 con 0x7f58a4163650 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c011850 con 0x7f58a4163650 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a40ce900 con 0x7f58a4163650 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c011c40 con 0x7f58a4163650 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.575+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41e9c00 con 0x7f58a4163650 2024-04-26T20:01:01.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.576+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a40693c0 con 0x7f58a4163650 2024-04-26T20:01:01.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.577+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f589c018020 con 0x7f58a4163650 2024-04-26T20:01:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.578+0000 7f589b400640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 0x7f588006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.578+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f589c08b710 con 0x7f58a4163650 2024-04-26T20:01:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.579+0000 7f58a3400640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 0x7f588006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:01.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.579+0000 7f58a3400640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 0x7f588006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5894034000 tx=0x7f589402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.583+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f589c01f020 con 0x7f58a4163650 2024-04-26T20:01:01.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.781+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a408d090 con 0x7f58a4163650 2024-04-26T20:01:01.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.783+0000 7f589b400640 1 -- 172.21.15.88:0/3647859880 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12879 (secure 0 0 0) 0x7f589c0172c0 con 0x7f58a4163650 2024-04-26T20:01:01.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:01.784 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:00:57.809118+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:01.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.785+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 msgr2=0x7f588006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.785+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 0x7f588006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5894034000 tx=0x7f589402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.785+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 msgr2=0x7f58a40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.785+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b410 tx=0x7f589c00b810 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 shutdown_connections 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a4163650 0x7f58a40ce2a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58a4162b00 0x7f58a40cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f588006c8f0 0x7f588006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 --2- 172.21.15.88:0/3647859880 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f58a415f1b0 0x7f58a415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 >> 172.21.15.88:0/3647859880 conn(0x7f58a409e230 msgr2=0x7f58a40b8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 shutdown_connections 2024-04-26T20:01:01.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:01.786+0000 7f58a8a00640 1 -- 172.21.15.88:0/3647859880 wait complete. 2024-04-26T20:01:01.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:01.806 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.008+0000 7fd881a00640 1 Processor -- start 2024-04-26T20:01:02.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.008+0000 7fd881a00640 1 -- start start 2024-04-26T20:01:02.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.009+0000 7fd881a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 0x7fd87c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.009+0000 7fd881a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 0x7fd87c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd881a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87c0b4d20 con 0x7fd87c160d60 2024-04-26T20:01:02.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd881a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd87c146c30 con 0x7fd87c161140 2024-04-26T20:01:02.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd881a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd87c0b5000 con 0x7fd87c146280 2024-04-26T20:01:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd881000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 0x7fd87c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd873e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 0x7fd87c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd873e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 0x7fd87c06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:56208/0 (socket says 172.21.15.88:56208) 2024-04-26T20:01:02.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.010+0000 7fd873e00640 1 -- 172.21.15.88:0/1359334945 learned_addr learned my addr 172.21.15.88:0/1359334945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.011+0000 7fd881000640 1 -- 172.21.15.88:0/1359334945 >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 msgr2=0x7fd87c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.011+0000 7fd881000640 1 --2- 172.21.15.88:0/1359334945 >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 0x7fd87c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.011+0000 7fd881000640 1 -- 172.21.15.88:0/1359334945 >> v1:172.21.15.132:6789/0 conn(0x7fd87c160d60 legacy=0x7fd87c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.011+0000 7fd881000640 1 -- 172.21.15.88:0/1359334945 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c0b52e0 con 0x7fd87c146280 2024-04-26T20:01:02.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.013+0000 7fd881000640 1 --2- 172.21.15.88:0/1359334945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 0x7fd87c1466f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd874009dc0 tx=0x7fd87402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5448dc7da60c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.013+0000 7fd87b400640 1 -- 172.21.15.88:0/1359334945 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87402ae60 con 0x7fd87c146280 2024-04-26T20:01:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.013+0000 7fd87b400640 1 -- 172.21.15.88:0/1359334945 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd874007740 con 0x7fd87c146280 2024-04-26T20:01:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.013+0000 7fd87b400640 1 -- 172.21.15.88:0/1359334945 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874007b10 con 0x7fd87c146280 2024-04-26T20:01:02.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 -- 172.21.15.88:0/1359334945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 msgr2=0x7fd87c1466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 --2- 172.21.15.88:0/1359334945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 0x7fd87c1466f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd874009dc0 tx=0x7fd87402a680 comp rx=0 tx=0).stop 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 -- 172.21.15.88:0/1359334945 shutdown_connections 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 --2- 172.21.15.88:0/1359334945 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c146280 0x7fd87c1466f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 --2- 172.21.15.88:0/1359334945 >> v2:172.21.15.132:3300/0 conn(0x7fd87c161140 0x7fd87c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 -- 172.21.15.88:0/1359334945 >> 172.21.15.88:0/1359334945 conn(0x7fd87c0b2900 msgr2=0x7fd87c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 -- 172.21.15.88:0/1359334945 shutdown_connections 2024-04-26T20:01:02.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.014+0000 7fd881a00640 1 -- 172.21.15.88:0/1359334945 wait complete. 2024-04-26T20:01:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.016+0000 7fd881a00640 1 Processor -- start 2024-04-26T20:01:02.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.016+0000 7fd881a00640 1 -- start start 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd87c160d60 0x7fd87c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c161140 0x7fd87c136950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c06e420 con 0x7fd87c146280 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c0cf6f0 con 0x7fd87c160d60 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd87c06e5a0 con 0x7fd87c161140 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd87be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd87be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52906/0 (socket says 172.21.15.88:52906) 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd87be00640 1 -- 172.21.15.88:0/1093537894 learned_addr learned my addr 172.21.15.88:0/1093537894 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd873e00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd87c160d60 0x7fd87c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd87be00640 1 -- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c161140 msgr2=0x7fd87c136950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd881000640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c161140 0x7fd87c136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.017+0000 7fd87be00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c161140 0x7fd87c136950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd87be00640 1 -- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd87c160d60 msgr2=0x7fd87c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd87be00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd87c160d60 0x7fd87c15c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd87be00640 1 -- 172.21.15.88:0/1093537894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c159070 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd87be00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c400 tx=0x7fd86c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0052d0 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86c005490 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c007260 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c13b870 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.018+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c13be30 con 0x7fd87c146280 2024-04-26T20:01:02.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.019+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c146050 con 0x7fd87c146280 2024-04-26T20:01:02.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.020+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd86c007400 con 0x7fd87c146280 2024-04-26T20:01:02.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.021+0000 7fd878c00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 0x7fd85806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.021+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd86c08adb0 con 0x7fd87c146280 2024-04-26T20:01:02.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.022+0000 7fd873e00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 0x7fd85806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.022+0000 7fd873e00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 0x7fd85806ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd87c159a00 tx=0x7fd864008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.026+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd87c146050 con 0x7fd87c146280 2024-04-26T20:01:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.210+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd87c0cee70 con 0x7fd87c146280 2024-04-26T20:01:02.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.211+0000 7fd878c00640 1 -- 172.21.15.88:0/1093537894 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fd87c0cee70 con 0x7fd87c146280 2024-04-26T20:01:02.212 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:02.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 msgr2=0x7fd85806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 0x7fd85806ede0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd87c159a00 tx=0x7fd864008040 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 msgr2=0x7fd87c15bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c400 tx=0x7fd86c00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 shutdown_connections 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd85806c920 0x7fd85806ede0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd87c161140 0x7fd87c136950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd87c160d60 0x7fd87c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 --2- 172.21.15.88:0/1093537894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd87c146280 0x7fd87c15bd20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.213+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 >> 172.21.15.88:0/1093537894 conn(0x7fd87c0b2900 msgr2=0x7fd87c1673c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.214+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 shutdown_connections 2024-04-26T20:01:02.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.214+0000 7fd881a00640 1 -- 172.21.15.88:0/1093537894 wait complete. 2024-04-26T20:01:02.232 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:01:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.429+0000 7f65e9a00640 1 Processor -- start 2024-04-26T20:01:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.429+0000 7f65e9a00640 1 -- start start 2024-04-26T20:01:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.430+0000 7f65e9a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f65e4147d50 0x7f65e40cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.430+0000 7f65e9a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f65e40ca540 0x7f65e4168b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.430+0000 7f65e9a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e406e680 con 0x7f65e40cc240 2024-04-26T20:01:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.430+0000 7f65e9a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f65e40cc6a0 con 0x7f65e40ca540 2024-04-26T20:01:02.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.430+0000 7f65e9a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f65e4070130 con 0x7f65e4147d50 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f65e40ca540 0x7f65e4168b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f65e40ca540 0x7f65e4168b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52920/0 (socket says 172.21.15.88:52920) 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 -- 172.21.15.88:0/3920724282 learned_addr learned my addr 172.21.15.88:0/3920724282 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e3e00640 1 --2- 172.21.15.88:0/3920724282 >> v2:172.21.15.132:3300/0 conn(0x7f65e4147d50 0x7f65e40cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 -- 172.21.15.88:0/3920724282 >> v2:172.21.15.132:3300/0 conn(0x7f65e4147d50 msgr2=0x7f65e40cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 --2- 172.21.15.88:0/3920724282 >> v2:172.21.15.132:3300/0 conn(0x7f65e4147d50 0x7f65e40cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 -- 172.21.15.88:0/3920724282 >> v1:172.21.15.132:6789/0 conn(0x7f65e40cc240 legacy=0x7f65e40c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.431+0000 7f65e9000640 1 -- 172.21.15.88:0/3920724282 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e40caa40 con 0x7f65e40ca540 2024-04-26T20:01:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.432+0000 7f65e9000640 1 --2- 172.21.15.88:0/3920724282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e4168b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65e4050f80 tx=0x7f65dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd70c61377f80ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e3400640 1 -- 172.21.15.88:0/3920724282 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc036070 con 0x7f65e40ca540 2024-04-26T20:01:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e3400640 1 -- 172.21.15.88:0/3920724282 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65dc031600 con 0x7f65e40ca540 2024-04-26T20:01:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e3400640 1 -- 172.21.15.88:0/3920724282 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0319d0 con 0x7f65e40ca540 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 -- 172.21.15.88:0/3920724282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 msgr2=0x7f65e4168b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 --2- 172.21.15.88:0/3920724282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e4168b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65e4050f80 tx=0x7f65dc02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 -- 172.21.15.88:0/3920724282 shutdown_connections 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 --2- 172.21.15.88:0/3920724282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e4168b60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 --2- 172.21.15.88:0/3920724282 >> v2:172.21.15.132:3300/0 conn(0x7f65e4147d50 0x7f65e40cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 -- 172.21.15.88:0/3920724282 >> 172.21.15.88:0/3920724282 conn(0x7f65e40d0ae0 msgr2=0x7f65e40d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 -- 172.21.15.88:0/3920724282 shutdown_connections 2024-04-26T20:01:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.433+0000 7f65e9a00640 1 -- 172.21.15.88:0/3920724282 wait complete. 2024-04-26T20:01:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.435+0000 7f65e9a00640 1 Processor -- start 2024-04-26T20:01:02.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.435+0000 7f65e9a00640 1 -- start start 2024-04-26T20:01:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 0x7f65e40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65e4147d50 0x7f65e40b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65e406ec00 con 0x7f65e40ca540 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f65e406e920 con 0x7f65e40cc240 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e9a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f65e40ce6b0 con 0x7f65e4147d50 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52930/0 (socket says 172.21.15.88:52930) 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.436+0000 7f65e3e00640 1 -- 172.21.15.88:0/2778108843 learned_addr learned my addr 172.21.15.88:0/2778108843 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65db400640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 0x7f65e40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e9000640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65e4147d50 0x7f65e40b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 -- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65e4147d50 msgr2=0x7f65e40b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65e4147d50 0x7f65e40b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 -- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 msgr2=0x7f65e40b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 0x7f65e40b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 -- 172.21.15.88:0/2778108843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e41403c0 con 0x7f65e40ca540 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65db400640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 0x7f65e40b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e3e00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c7f0 tx=0x7f65d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.437+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4015070 con 0x7f65e40ca540 2024-04-26T20:01:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.438+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4004240 con 0x7f65e40ca540 2024-04-26T20:01:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.438+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e41406a0 con 0x7f65e40ca540 2024-04-26T20:01:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.438+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007620 con 0x7f65e40ca540 2024-04-26T20:01:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.438+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e4140c60 con 0x7f65e40ca540 2024-04-26T20:01:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.439+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e40693c0 con 0x7f65e40ca540 2024-04-26T20:01:02.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.440+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65d40079d0 con 0x7f65e40ca540 2024-04-26T20:01:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.441+0000 7f65e0c00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 0x7f65c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.441+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f65d408bd30 con 0x7f65e40ca540 2024-04-26T20:01:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.441+0000 7f65db400640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 0x7f65c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.442+0000 7f65db400640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 0x7f65c006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65e40722d0 tx=0x7f65cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.446+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65d405a410 con 0x7f65e40ca540 2024-04-26T20:01:02.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.647+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f65e413f790 con 0x7f65e40ca540 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.648+0000 7f65e0c00640 1 -- 172.21.15.88:0/2778108843 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f65d4059d10 con 0x7f65e40ca540 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:02.649 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 msgr2=0x7f65c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 0x7f65c006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65e40722d0 tx=0x7f65cc008040 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 msgr2=0x7f65e413e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65d400c7f0 tx=0x7f65d400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 shutdown_connections 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f65c006c8f0 0x7f65c006edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f65e4147d50 0x7f65e40b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65e40cc240 0x7f65e40b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 --2- 172.21.15.88:0/2778108843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65e40ca540 0x7f65e413e700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 >> 172.21.15.88:0/2778108843 conn(0x7f65e40d0ae0 msgr2=0x7f65e4072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 shutdown_connections 2024-04-26T20:01:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.651+0000 7f65e9a00640 1 -- 172.21.15.88:0/2778108843 wait complete. 2024-04-26T20:01:02.669 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:01:02.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:01:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.858+0000 7fcf21400640 1 Processor -- start 2024-04-26T20:01:02.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.858+0000 7fcf21400640 1 -- start start 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.859+0000 7fcf21400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 0x7fcf1c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.859+0000 7fcf21400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c0ae160 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.859+0000 7fcf21400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c06ff80 con 0x7fcf1c15d170 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.859+0000 7fcf21400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c15d550 con 0x7fcf1c0aff10 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fcf1c0cffe0 0x7fcf1c0af800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50108/0 (socket says 172.21.15.88:50108) 2024-04-26T20:01:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19e00640 1 -- 172.21.15.88:0/2100184295 learned_addr learned my addr 172.21.15.88:0/2100184295 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186904049 0 0) 0x7fcf1c0ae160 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf1b200640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 0x7fcf1c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcefc003520 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251822098 0 0) 0x7fcefc003520 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 msgr2=0x7fcf1c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 0x7fcf1c072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 >> v1:172.21.15.132:6789/0 conn(0x7fcf1c15d170 legacy=0x7fcf1c06ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.860+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c070260 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fcf040019f0 con 0x7fcf1c0cffe0 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 >> v1:172.21.15.88:6789/0 conn(0x7fcf1c0cffe0 legacy=0x7fcf1c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcefc004800 0x7fcefc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 0x7fcefc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c15d550 con 0x7fcefc0039e0 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc00bac0 con 0x7fcefc004800 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc00bcc0 con 0x7fcf1c0aff10 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 -- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 msgr2=0x7fcefc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcf1c0aff10 0x7fcefc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 -- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcefc004800 msgr2=0x7fcefc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcefc004800 0x7fcefc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.861+0000 7fcf1a800640 1 -- 172.21.15.88:0/2100184295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc003520 con 0x7fcefc0039e0 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.862+0000 7fcf1a800640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c001750 tx=0x7fcefc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e5639fea4c65a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.862+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c001370 con 0x7fcefc0039e0 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.862+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0052d0 con 0x7fcefc0039e0 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf19400640 1 -- 172.21.15.88:0/2100184295 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c02abe0 con 0x7fcefc0039e0 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 -- 172.21.15.88:0/2100184295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcefc0039e0 msgr2=0x7fcefc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 --2- 172.21.15.88:0/2100184295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcefc0039e0 0x7fcefc003e20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c001750 tx=0x7fcefc005830 comp rx=0 tx=0).stop 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf1b200640 1 -- 172.21.15.88:0/2100184295 reap_dead start 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 -- 172.21.15.88:0/2100184295 shutdown_connections 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 -- 172.21.15.88:0/2100184295 >> 172.21.15.88:0/2100184295 conn(0x7fcf1c0d0c40 msgr2=0x7fcf1c0d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 -- 172.21.15.88:0/2100184295 shutdown_connections 2024-04-26T20:01:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.863+0000 7fcf21400640 1 -- 172.21.15.88:0/2100184295 wait complete. 2024-04-26T20:01:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.865+0000 7fcf21400640 1 Processor -- start 2024-04-26T20:01:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.865+0000 7fcf21400640 1 -- start start 2024-04-26T20:01:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcefc0039e0 0x7fcf1c14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1c14b350 0x7fcf1c09dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0d0440 con 0x7fcf1c09e250 2024-04-26T20:01:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c06fd30 con 0x7fcf1c14b350 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf21400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0add80 con 0x7fcefc0039e0 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf1b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52950/0 (socket says 172.21.15.88:52950) 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.866+0000 7fcf1b200640 1 -- 172.21.15.88:0/3346518004 learned_addr learned my addr 172.21.15.88:0/3346518004 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 -- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcefc0039e0 msgr2=0x7fcf1c14ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcefc0039e0 0x7fcf1c14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 -- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1c14b350 msgr2=0x7fcf1c09dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1c14b350 0x7fcf1c09dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:02.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 -- 172.21.15.88:0/3346518004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c09baf0 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf1b200640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c06dca0 tx=0x7fcf1400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14017070 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1400c680 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c15e2e0 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.867+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400ca70 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.868+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c159c90 con 0x7fcf1c09e250 2024-04-26T20:01:02.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.869+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c069650 con 0x7fcf1c09e250 2024-04-26T20:01:02.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.870+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf1401a070 con 0x7fcf1c09e250 2024-04-26T20:01:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.870+0000 7fcf12a00640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 0x7fcef806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.871+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcf1408b200 con 0x7fcf1c09e250 2024-04-26T20:01:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.871+0000 7fcf1a800640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 0x7fcef806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:02.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.872+0000 7fcf1a800640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 0x7fcef806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c032000 tx=0x7fcf0c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:02.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:02.875+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf14059970 con 0x7fcf1c09e250 2024-04-26T20:01:03.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.075+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcf1c0b0550 con 0x7fcf1c09e250 2024-04-26T20:01:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.856+0000 7fcf12a00640 1 -- 172.21.15.88:0/3346518004 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fcf14059270 con 0x7fcf1c09e250 2024-04-26T20:01:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-04-26T20:01:03.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 msgr2=0x7fcef806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 0x7fcef806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c032000 tx=0x7fcf0c02c000 comp rx=0 tx=0).stop 2024-04-26T20:01:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 msgr2=0x7fcf1c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:03.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c06dca0 tx=0x7fcf1400ec00 comp rx=0 tx=0).stop 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 shutdown_connections 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf1c09e250 0x7fcf1c15d6d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf1c14b350 0x7fcf1c09dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcef806c920 0x7fcef806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 --2- 172.21.15.88:0/3346518004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcefc0039e0 0x7fcf1c14ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 >> 172.21.15.88:0/3346518004 conn(0x7fcf1c0d0c40 msgr2=0x7fcf1c0cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 shutdown_connections 2024-04-26T20:01:03.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:03.861+0000 7fcf21400640 1 -- 172.21.15.88:0/3346518004 wait complete. 2024-04-26T20:01:03.888 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:01:04.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.095+0000 7f24ca000640 1 Processor -- start 2024-04-26T20:01:04.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.095+0000 7f24ca000640 1 -- start start 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24ca000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 0x7f24c4169f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24ca000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c406e0b0 con 0x7f24c415f4c0 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24ca000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c406e390 con 0x7f24c40703f0 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24ca000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f24c415f8a0 con 0x7f24c40cfe80 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24c3400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f24c415f4c0 0x7f24c406d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50116/0 (socket says 172.21.15.88:50116) 2024-04-26T20:01:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.097+0000 7f24c3400640 1 -- 172.21.15.88:0/157957325 learned_addr learned my addr 172.21.15.88:0/157957325 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:04.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c3e00640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 0x7f24c4169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310670985 0 0) 0x7f24c406e0b0 con 0x7f24c415f4c0 2024-04-26T20:01:04.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a8003520 con 0x7f24c415f4c0 2024-04-26T20:01:04.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3468252636 0 0) 0x7f24a8003520 con 0x7f24c415f4c0 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 msgr2=0x7f24c4169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 0x7f24c4169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 >> v1:172.21.15.132:6789/0 conn(0x7f24c40703f0 legacy=0x7f24c40cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c406e670 con 0x7f24c415f4c0 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f24b4001670 con 0x7f24c415f4c0 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 >> v1:172.21.15.88:6789/0 conn(0x7f24c415f4c0 legacy=0x7f24c406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a80039e0 0x7f24a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.098+0000 7f24c2000640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2000640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 0x7f24a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24c415f8a0 con 0x7f24a8004800 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f24a800bad0 con 0x7f24a80039e0 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f24a800bcd0 con 0x7f24c40cfe80 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c3400640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a80039e0 0x7f24a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 -- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 msgr2=0x7f24a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cfe80 0x7f24a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 -- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a80039e0 msgr2=0x7f24a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a80039e0 0x7f24a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c2a00640 1 -- 172.21.15.88:0/157957325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a8003520 con 0x7f24a8004800 2024-04-26T20:01:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.099+0000 7f24c3400640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a80039e0 0x7f24a8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.100+0000 7f24c2a00640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24b0001720 tx=0x7f24a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764524f011e53eef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0001370 con 0x7f24a8004800 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b00052d0 con 0x7f24a8004800 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24c2000640 1 -- 172.21.15.88:0/157957325 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b002ab90 con 0x7f24a8004800 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 -- 172.21.15.88:0/157957325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 msgr2=0x7f24a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 --2- 172.21.15.88:0/157957325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f24b0001720 tx=0x7f24a8005830 comp rx=0 tx=0).stop 2024-04-26T20:01:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24c3e00640 1 -- 172.21.15.88:0/157957325 reap_dead start 2024-04-26T20:01:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 -- 172.21.15.88:0/157957325 shutdown_connections 2024-04-26T20:01:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 -- 172.21.15.88:0/157957325 >> 172.21.15.88:0/157957325 conn(0x7f24c4075f80 msgr2=0x7f24c40d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 -- 172.21.15.88:0/157957325 shutdown_connections 2024-04-26T20:01:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.101+0000 7f24ca000640 1 -- 172.21.15.88:0/157957325 wait complete. 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.103+0000 7f24ca000640 1 Processor -- start 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.103+0000 7f24ca000640 1 -- start start 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 0x7f24c40cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cb260 0x7f24c40b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40d2620 con 0x7f24a8004800 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40d1080 con 0x7f24c41f8e20 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24ca000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f24c40d27a0 con 0x7f24c40cb260 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24c3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24c3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52966/0 (socket says 172.21.15.88:52966) 2024-04-26T20:01:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.104+0000 7f24c3400640 1 -- 172.21.15.88:0/1870922270 learned_addr learned my addr 172.21.15.88:0/1870922270 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:04.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c2a00640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 0x7f24c40cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 -- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cb260 msgr2=0x7f24c40b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cb260 0x7f24c40b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 -- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 msgr2=0x7f24c40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 0x7f24c40cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 -- 172.21.15.88:0/1870922270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c40cb6e0 con 0x7f24a8004800 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c2a00640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 0x7f24c40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24c3400640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f24b400d790 tx=0x7f24b400dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4019070 con 0x7f24a8004800 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b40074e0 con 0x7f24a8004800 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b40049e0 con 0x7f24a8004800 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.105+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41f8340 con 0x7f24a8004800 2024-04-26T20:01:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.106+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41f8900 con 0x7f24a8004800 2024-04-26T20:01:04.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.107+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c4069650 con 0x7f24a8004800 2024-04-26T20:01:04.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.108+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24b401c070 con 0x7f24a8004800 2024-04-26T20:01:04.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.109+0000 7f24bb400640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:04.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.109+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f24b408b3f0 con 0x7f24a8004800 2024-04-26T20:01:04.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.109+0000 7f24c2a00640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:04.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.110+0000 7f24c2a00640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 0x7f24a406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24b0032000 tx=0x7f24b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:04.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.114+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24b40599f0 con 0x7f24a8004800 2024-04-26T20:01:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.315+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f24c40d0880 con 0x7f24a8004800 2024-04-26T20:01:04.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.863+0000 7f24bb400640 1 -- 172.21.15.88:0/1870922270 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f24b40592f0 con 0x7f24a8004800 2024-04-26T20:01:04.864 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-04-26T20:01:04.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.867+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 msgr2=0x7f24a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.867+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 0x7f24a406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24b0032000 tx=0x7f24b002e000 comp rx=0 tx=0).stop 2024-04-26T20:01:04.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.867+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 msgr2=0x7f24c40a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:04.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.867+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f24b400d790 tx=0x7f24b400dc50 comp rx=0 tx=0).stop 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 shutdown_connections 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f24a406c8f0 0x7f24a406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f24c40cb260 0x7f24c40b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24c41f8e20 0x7f24c40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 --2- 172.21.15.88:0/1870922270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a8004800 0x7f24c40a06a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:04.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.868+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 >> 172.21.15.88:0/1870922270 conn(0x7f24c4075f80 msgr2=0x7f24c40d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:04.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.869+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 shutdown_connections 2024-04-26T20:01:04.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:04.871+0000 7f24ca000640 1 -- 172.21.15.88:0/1870922270 wait complete. 2024-04-26T20:01:04.904 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:01:05.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.081+0000 7fbdd4e00640 1 Processor -- start 2024-04-26T20:01:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.081+0000 7fbdd4e00640 1 -- start start 2024-04-26T20:01:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.082+0000 7fbdd4e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbdd00b6170 0x7fbdd00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.082+0000 7fbdd4e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00d3830 con 0x7fbdd00d19c0 2024-04-26T20:01:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.082+0000 7fbdd4e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00d3b10 con 0x7fbdd00d1da0 2024-04-26T20:01:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.082+0000 7fbdd4e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd00d40d0 con 0x7fbdd00b6170 2024-04-26T20:01:05.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcde00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fbdd00d19c0 0x7fbdd00d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50122/0 (socket says 172.21.15.88:50122) 2024-04-26T20:01:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcde00640 1 -- 172.21.15.88:0/2513224776 learned_addr learned my addr 172.21.15.88:0/2513224776 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcd400640 1 --2- 172.21.15.88:0/2513224776 >> v2:172.21.15.88:3300/0 conn(0x7fbdd00b6170 0x7fbdd00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcca00640 1 -- 172.21.15.88:0/2513224776 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596902908 0 0) 0x7fbdd00d3830 con 0x7fbdd00d19c0 2024-04-26T20:01:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcca00640 1 -- 172.21.15.88:0/2513224776 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb0003520 con 0x7fbdd00d19c0 2024-04-26T20:01:05.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcd400640 1 -- 172.21.15.88:0/2513224776 >> v1:172.21.15.132:6789/0 conn(0x7fbdd00d1da0 legacy=0x7fbdd0162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcd400640 1 -- 172.21.15.88:0/2513224776 >> v1:172.21.15.88:6789/0 conn(0x7fbdd00d19c0 legacy=0x7fbdd00d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.083+0000 7fbdcd400640 1 -- 172.21.15.88:0/2513224776 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd00d3df0 con 0x7fbdd00b6170 2024-04-26T20:01:05.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.085+0000 7fbdcd400640 1 --2- 172.21.15.88:0/2513224776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00c8c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097a0 tx=0x7fbdb802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b34fa101947283e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.085+0000 7fbdcca00640 1 -- 172.21.15.88:0/2513224776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8037070 con 0x7fbdd00b6170 2024-04-26T20:01:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.085+0000 7fbdcca00640 1 -- 172.21.15.88:0/2513224776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb802ae50 con 0x7fbdd00b6170 2024-04-26T20:01:05.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.085+0000 7fbdcca00640 1 -- 172.21.15.88:0/2513224776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8032660 con 0x7fbdd00b6170 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2513224776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 msgr2=0x7fbdd00c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2513224776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00c8c70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097a0 tx=0x7fbdb802a140 comp rx=0 tx=0).stop 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2513224776 shutdown_connections 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2513224776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00c8c70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2513224776 >> 172.21.15.88:0/2513224776 conn(0x7fbdd009e460 msgr2=0x7fbdd00d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2513224776 shutdown_connections 2024-04-26T20:01:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.086+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2513224776 wait complete. 2024-04-26T20:01:05.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.088+0000 7fbdd4e00640 1 Processor -- start 2024-04-26T20:01:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.088+0000 7fbdd4e00640 1 -- start start 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.089+0000 7fbdd4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.089+0000 7fbdd4e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd00d19c0 0x7fbdd0153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdd4e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 0x7fbdd006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdd4e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0162d50 con 0x7fbdd00b6170 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdd4e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd00cfc30 con 0x7fbdd00d19c0 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdd4e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd0162ed0 con 0x7fbdd00d1da0 2024-04-26T20:01:05.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52986/0 (socket says 172.21.15.88:52986) 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 -- 172.21.15.88:0/2757240968 learned_addr learned my addr 172.21.15.88:0/2757240968 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcd400640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd00d19c0 0x7fbdd0153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdce800640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 0x7fbdd006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 -- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 msgr2=0x7fbdd006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 0x7fbdd006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 -- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd00d19c0 msgr2=0x7fbdd0153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd00d19c0 0x7fbdd0153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdcde00640 1 -- 172.21.15.88:0/2757240968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd00baa50 con 0x7fbdd00b6170 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.090+0000 7fbdce800640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 0x7fbdd006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:05.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdcde00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c3c0 tx=0x7fbdc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:05.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0017070 con 0x7fbdd00b6170 2024-04-26T20:01:05.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0005ce0 con 0x7fbdd00b6170 2024-04-26T20:01:05.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd0154470 con 0x7fbdd00b6170 2024-04-26T20:01:05.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00072f0 con 0x7fbdd00b6170 2024-04-26T20:01:05.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.091+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01f8bc0 con 0x7fbdd00b6170 2024-04-26T20:01:05.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.092+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd0069650 con 0x7fbdd00b6170 2024-04-26T20:01:05.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.093+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdc001a070 con 0x7fbdd00b6170 2024-04-26T20:01:05.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.094+0000 7fbdc6000640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 0x7fbdac06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:05.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.094+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fbdc008ae70 con 0x7fbdd00b6170 2024-04-26T20:01:05.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.095+0000 7fbdcd400640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 0x7fbdac06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:05.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.095+0000 7fbdcd400640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 0x7fbdac06ee30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8034000 tx=0x7fbdb802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:05.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.099+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbdd00baa50 con 0x7fbdd00b6170 2024-04-26T20:01:05.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.340+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbdd00d26d0 con 0x7fbdd00b6170 2024-04-26T20:01:05.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.911+0000 7fbdc6000640 1 -- 172.21.15.88:0/2757240968 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fbdc005f020 con 0x7fbdd00b6170 2024-04-26T20:01:05.917 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T20:01:05.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.921+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 msgr2=0x7fbdac06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.921+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 0x7fbdac06ee30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8034000 tx=0x7fbdb802e000 comp rx=0 tx=0).stop 2024-04-26T20:01:05.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.921+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 msgr2=0x7fbdd00ba510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:05.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.921+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c3c0 tx=0x7fbdc000c880 comp rx=0 tx=0).stop 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 shutdown_connections 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbdac06c970 0x7fbdac06ee30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbdd00d1da0 0x7fbdd006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdd00d19c0 0x7fbdd0153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 --2- 172.21.15.88:0/2757240968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdd00b6170 0x7fbdd00ba510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:05.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.922+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 >> 172.21.15.88:0/2757240968 conn(0x7fbdd009e460 msgr2=0x7fbdd00c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:05.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.923+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 shutdown_connections 2024-04-26T20:01:05.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:05.923+0000 7fbdd4e00640 1 -- 172.21.15.88:0/2757240968 wait complete. 2024-04-26T20:01:05.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:06.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.120+0000 7f581be00640 1 Processor -- start 2024-04-26T20:01:06.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.120+0000 7f581be00640 1 -- start start 2024-04-26T20:01:06.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f581be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f581406c770 0x7f581406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f581be00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f581415cfd0 0x7f5814073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f581be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581406d090 con 0x7f581415cb70 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f581be00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f58140af460 con 0x7f581415cfd0 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f581be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f58140ae850 con 0x7f581406c770 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f5819e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f581415cfd0 0x7f5814073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f5819e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f581415cfd0 0x7f5814073830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52990/0 (socket says 172.21.15.88:52990) 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f5819e00640 1 -- 172.21.15.88:0/200424478 learned_addr learned my addr 172.21.15.88:0/200424478 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:06.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.122+0000 7f5819400640 1 --2- 172.21.15.88:0/200424478 >> v2:172.21.15.132:3300/0 conn(0x7f581406c770 0x7f581406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.123+0000 7f5819e00640 1 -- 172.21.15.88:0/200424478 >> v2:172.21.15.132:3300/0 conn(0x7f581406c770 msgr2=0x7f581406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.123+0000 7f5819e00640 1 --2- 172.21.15.88:0/200424478 >> v2:172.21.15.132:3300/0 conn(0x7f581406c770 0x7f581406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.123+0000 7f5819e00640 1 -- 172.21.15.88:0/200424478 >> v1:172.21.15.132:6789/0 conn(0x7f581415cb70 legacy=0x7f5814073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.123+0000 7f5819e00640 1 -- 172.21.15.88:0/200424478 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58140aeb30 con 0x7f581415cfd0 2024-04-26T20:01:06.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.124+0000 7f5819e00640 1 --2- 172.21.15.88:0/200424478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cfd0 0x7f5814073830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58140741f0 tx=0x7f580c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4564c1a89e3a81fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.124+0000 7f5813e00640 1 -- 172.21.15.88:0/200424478 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c037070 con 0x7f581415cfd0 2024-04-26T20:01:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.124+0000 7f5813e00640 1 -- 172.21.15.88:0/200424478 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580c032440 con 0x7f581415cfd0 2024-04-26T20:01:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.124+0000 7f5813e00640 1 -- 172.21.15.88:0/200424478 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c032810 con 0x7f581415cfd0 2024-04-26T20:01:06.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 -- 172.21.15.88:0/200424478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cfd0 msgr2=0x7f5814073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 --2- 172.21.15.88:0/200424478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cfd0 0x7f5814073830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58140741f0 tx=0x7f580c02a230 comp rx=0 tx=0).stop 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 -- 172.21.15.88:0/200424478 shutdown_connections 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 --2- 172.21.15.88:0/200424478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cfd0 0x7f5814073830 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 --2- 172.21.15.88:0/200424478 >> v2:172.21.15.132:3300/0 conn(0x7f581406c770 0x7f581406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 -- 172.21.15.88:0/200424478 >> 172.21.15.88:0/200424478 conn(0x7f58140b2d50 msgr2=0x7f58140b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 -- 172.21.15.88:0/200424478 shutdown_connections 2024-04-26T20:01:06.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.125+0000 7f581be00640 1 -- 172.21.15.88:0/200424478 wait complete. 2024-04-26T20:01:06.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.126+0000 7f581be00640 1 Processor -- start 2024-04-26T20:01:06.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 -- start start 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581406c770 0x7f581414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 0x7f581415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f581415c600 con 0x7f581415cb70 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f58140af0f0 con 0x7f581415cfd0 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.127+0000 7f581be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f581415c780 con 0x7f581406c770 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52998/0 (socket says 172.21.15.88:52998) 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 -- 172.21.15.88:0/3118344170 learned_addr learned my addr 172.21.15.88:0/3118344170 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5819400640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581406c770 0x7f581414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5819e00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 0x7f581415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 -- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581406c770 msgr2=0x7f581414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581406c770 0x7f581414abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 -- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 msgr2=0x7f581415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 0x7f581415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5818a00640 1 -- 172.21.15.88:0/3118344170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581414d6b0 con 0x7f581415cb70 2024-04-26T20:01:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.128+0000 7f5819e00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 0x7f581415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f5818a00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f580000c410 tx=0x7f580000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800017070 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58000035f0 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800003a20 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581414d990 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.129+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140ce020 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.130+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814069540 con 0x7f581415cb70 2024-04-26T20:01:06.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.131+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5800003bc0 con 0x7f581415cb70 2024-04-26T20:01:06.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.132+0000 7f5811600640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 0x7f57f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.132+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f580008ae40 con 0x7f581415cb70 2024-04-26T20:01:06.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.132+0000 7f5819400640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 0x7f57f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.133+0000 7f5819400640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 0x7f57f406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5804009f10 tx=0x7f580400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.137+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5800059270 con 0x7f581415cb70 2024-04-26T20:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.321+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f581406f400 con 0x7f581415cb70 2024-04-26T20:01:06.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.323+0000 7f5811600640 1 -- 172.21.15.88:0/3118344170 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7f58000010c0 con 0x7f581415cb70 2024-04-26T20:01:06.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:06.324 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":32,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:05.341780+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 msgr2=0x7f57f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 0x7f57f406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5804009f10 tx=0x7f580400b040 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 msgr2=0x7f581415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f580000c410 tx=0x7f580000c8d0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 shutdown_connections 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f581415cfd0 0x7f581415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f581415cb70 0x7f581415abe0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f57f406c8f0 0x7f57f406edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 --2- 172.21.15.88:0/3118344170 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f581406c770 0x7f581414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 >> 172.21.15.88:0/3118344170 conn(0x7f58140b2d50 msgr2=0x7f58140b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:06.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 shutdown_connections 2024-04-26T20:01:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.326+0000 7f581be00640 1 -- 172.21.15.88:0/3118344170 wait complete. 2024-04-26T20:01:06.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:06.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.538+0000 7f2eada00640 1 Processor -- start 2024-04-26T20:01:06.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.538+0000 7f2eada00640 1 -- start start 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2eada00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2ea8163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2eada00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea80d3f50 con 0x7f2ea80b82b0 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2eada00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea80d36b0 con 0x7f2ea815f1b0 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2eada00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea80d3990 con 0x7f2ea80d5990 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2ead000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2ea80b82b0 0x7f2ea80b8740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50124/0 (socket says 172.21.15.88:50124) 2024-04-26T20:01:06.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.540+0000 7f2ead000640 1 -- 172.21.15.88:0/93182935 learned_addr learned my addr 172.21.15.88:0/93182935 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2e9f400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2ea8163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649719596 0 0) 0x7f2ea80d3f50 con 0x7f2ea80b82b0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e8c003520 con 0x7f2ea80b82b0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 832644983 0 0) 0x7f2e8c003520 con 0x7f2ea80b82b0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 msgr2=0x7f2ea8163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2ea8163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.541+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 >> v1:172.21.15.132:6789/0 conn(0x7f2ea815f1b0 legacy=0x7f2ea8162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea80d3c70 con 0x7f2ea80b82b0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f2ea00019f0 con 0x7f2ea80b82b0 2024-04-26T20:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 >> v1:172.21.15.88:6789/0 conn(0x7f2ea80b82b0 legacy=0x7f2ea80b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2e8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2e8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e8c004d40 0x7f2e8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea80d3990 con 0x7f2e8c0039e0 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c00baa0 con 0x7f2e8c004d40 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8c00bca0 con 0x7f2ea80d5990 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2e8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 -- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 msgr2=0x7f2e8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2e9f400640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2e8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80d5990 0x7f2e8c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 -- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e8c004d40 msgr2=0x7f2e8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e8c004d40 0x7f2e8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.542+0000 7f2ea7e00640 1 -- 172.21.15.88:0/93182935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c003520 con 0x7f2e8c0039e0 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.543+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2e8c003e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e98001720 tx=0x7f2e8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f490b43cbe3ade server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98001370 con 0x7f2e8c0039e0 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e980052f0 con 0x7f2e8c0039e0 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2ea7400640 1 -- 172.21.15.88:0/93182935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9802abc0 con 0x7f2e8c0039e0 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 -- 172.21.15.88:0/93182935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 msgr2=0x7f2e8c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 --2- 172.21.15.88:0/93182935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2e8c003e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2e98001720 tx=0x7f2e8c005830 comp rx=0 tx=0).stop 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2ead000640 1 -- 172.21.15.88:0/93182935 reap_dead start 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 -- 172.21.15.88:0/93182935 shutdown_connections 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 -- 172.21.15.88:0/93182935 >> 172.21.15.88:0/93182935 conn(0x7f2ea809e230 msgr2=0x7f2ea80d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 -- 172.21.15.88:0/93182935 shutdown_connections 2024-04-26T20:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.544+0000 7f2eada00640 1 -- 172.21.15.88:0/93182935 wait complete. 2024-04-26T20:01:06.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.546+0000 7f2eada00640 1 Processor -- start 2024-04-26T20:01:06.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.546+0000 7f2eada00640 1 -- start start 2024-04-26T20:01:06.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea814a290 0x7f2ea80b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80b9000 0x7f2ea80cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8163210 con 0x7f2e8c0039e0 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea80d33a0 con 0x7f2ea814a290 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2eada00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea8163390 con 0x7f2ea80b9000 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2ea7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53016/0 (socket says 172.21.15.88:53016) 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.547+0000 7f2ea7e00640 1 -- 172.21.15.88:0/2389168322 learned_addr learned my addr 172.21.15.88:0/2389168322 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 -- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80b9000 msgr2=0x7f2ea80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80b9000 0x7f2ea80cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 -- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea814a290 msgr2=0x7f2ea80b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2e9f400640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea814a290 0x7f2ea80b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea814a290 0x7f2ea80b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 -- 172.21.15.88:0/2389168322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea815f730 con 0x7f2e8c0039e0 2024-04-26T20:01:06.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea7e00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e98001840 tx=0x7f2e98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9803f020 con 0x7f2e8c0039e0 2024-04-26T20:01:06.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9802be50 con 0x7f2e8c0039e0 2024-04-26T20:01:06.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98007490 con 0x7f2e8c0039e0 2024-04-26T20:01:06.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.548+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea815fa10 con 0x7f2e8c0039e0 2024-04-26T20:01:06.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.549+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea815ffd0 con 0x7f2e8c0039e0 2024-04-26T20:01:06.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.550+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea80b9430 con 0x7f2e8c0039e0 2024-04-26T20:01:06.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.550+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e98042020 con 0x7f2e8c0039e0 2024-04-26T20:01:06.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.551+0000 7f2ea4c00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 0x7f2e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.551+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2e980ab6d0 con 0x7f2e8c0039e0 2024-04-26T20:01:06.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.552+0000 7f2e9f400640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 0x7f2e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.552+0000 7f2e9f400640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 0x7f2e8406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009830 tx=0x7f2e90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.556+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e98049030 con 0x7f2e8c0039e0 2024-04-26T20:01:06.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.757+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ea80693c0 con 0x7f2e8c0039e0 2024-04-26T20:01:06.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.758+0000 7f2ea4c00640 1 -- 172.21.15.88:0/2389168322 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16670 (secure 0 0 0) 0x7f2e98079a70 con 0x7f2e8c0039e0 2024-04-26T20:01:06.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:06.760 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":32,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:05.341780+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 msgr2=0x7f2e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 0x7f2e8406edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009830 tx=0x7f2e90008040 comp rx=0 tx=0).stop 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 msgr2=0x7f2ea80cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e98001840 tx=0x7f2e98001120 comp rx=0 tx=0).stop 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 shutdown_connections 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2e8406c8f0 0x7f2e8406edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ea80b9000 0x7f2ea80cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ea814a290 0x7f2ea80b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 --2- 172.21.15.88:0/2389168322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e8c0039e0 0x7f2ea80cbee0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 >> 172.21.15.88:0/2389168322 conn(0x7f2ea809e230 msgr2=0x7f2ea80b9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:06.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 shutdown_connections 2024-04-26T20:01:06.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.761+0000 7f2eada00640 1 -- 172.21.15.88:0/2389168322 wait complete. 2024-04-26T20:01:06.782 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:01:06.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.980+0000 7fc736800640 1 Processor -- start 2024-04-26T20:01:06.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.980+0000 7fc736800640 1 -- start start 2024-04-26T20:01:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.981+0000 7fc736800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc730161140 0x7fc73006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.981+0000 7fc736800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc730146280 0x7fc7301466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.981+0000 7fc736800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7300b4d20 con 0x7fc730160d60 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.981+0000 7fc736800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc730146c30 con 0x7fc730161140 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.981+0000 7fc736800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc7300b5000 con 0x7fc730146280 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc730161140 0x7fc73006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc735400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc730160d60 0x7fc7300b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50134/0 (socket says 172.21.15.88:50134) 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc730161140 0x7fc73006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53022/0 (socket says 172.21.15.88:53022) 2024-04-26T20:01:06.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc735400640 1 -- 172.21.15.88:0/757553830 learned_addr learned my addr 172.21.15.88:0/757553830 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc735e00640 1 --2- 172.21.15.88:0/757553830 >> v2:172.21.15.132:3300/0 conn(0x7fc730146280 0x7fc7301466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 -- 172.21.15.88:0/757553830 >> v2:172.21.15.132:3300/0 conn(0x7fc730146280 msgr2=0x7fc7301466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 --2- 172.21.15.88:0/757553830 >> v2:172.21.15.132:3300/0 conn(0x7fc730146280 0x7fc7301466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 -- 172.21.15.88:0/757553830 >> v1:172.21.15.88:6789/0 conn(0x7fc730160d60 legacy=0x7fc7300b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.982+0000 7fc734a00640 1 -- 172.21.15.88:0/757553830 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7300b52e0 con 0x7fc730161140 2024-04-26T20:01:06.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.983+0000 7fc734a00640 1 --2- 172.21.15.88:0/757553830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730161140 0x7fc73006dee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097a0 tx=0x7fc71c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4bfa747b237395 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc72fe00640 1 -- 172.21.15.88:0/757553830 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c037070 con 0x7fc730161140 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc72fe00640 1 -- 172.21.15.88:0/757553830 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c032440 con 0x7fc730161140 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc72fe00640 1 -- 172.21.15.88:0/757553830 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c032810 con 0x7fc730161140 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 -- 172.21.15.88:0/757553830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730161140 msgr2=0x7fc73006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 --2- 172.21.15.88:0/757553830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730161140 0x7fc73006dee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097a0 tx=0x7fc71c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 -- 172.21.15.88:0/757553830 shutdown_connections 2024-04-26T20:01:06.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 --2- 172.21.15.88:0/757553830 >> v2:172.21.15.132:3300/0 conn(0x7fc730146280 0x7fc7301466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 --2- 172.21.15.88:0/757553830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730161140 0x7fc73006dee0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.984+0000 7fc736800640 1 -- 172.21.15.88:0/757553830 >> 172.21.15.88:0/757553830 conn(0x7fc7300b2900 msgr2=0x7fc7300b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:06.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.985+0000 7fc736800640 1 -- 172.21.15.88:0/757553830 shutdown_connections 2024-04-26T20:01:06.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.985+0000 7fc736800640 1 -- 172.21.15.88:0/757553830 wait complete. 2024-04-26T20:01:06.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.986+0000 7fc736800640 1 Processor -- start 2024-04-26T20:01:06.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.986+0000 7fc736800640 1 -- start start 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc730146280 0x7fc73015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc730161140 0x7fc730139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc73006e420 con 0x7fc730160d60 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc7300cf6f0 con 0x7fc730146280 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.987+0000 7fc736800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc73006e5a0 con 0x7fc730161140 2024-04-26T20:01:06.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53026/0 (socket says 172.21.15.88:53026) 2024-04-26T20:01:06.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 -- 172.21.15.88:0/566075610 learned_addr learned my addr 172.21.15.88:0/566075610 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc735400640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc730146280 0x7fc73015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 -- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc730161140 msgr2=0x7fc730139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc730161140 0x7fc730139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 -- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc730146280 msgr2=0x7fc73015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc730146280 0x7fc73015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 -- 172.21.15.88:0/566075610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7300ca4c0 con 0x7fc730160d60 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.988+0000 7fc734a00640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc71c035040 tx=0x7fc71c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.989+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c037040 con 0x7fc730160d60 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.989+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c030770 con 0x7fc730160d60 2024-04-26T20:01:06.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.989+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc730072260 con 0x7fc730160d60 2024-04-26T20:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.989+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c030b80 con 0x7fc730160d60 2024-04-26T20:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.989+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc730072820 con 0x7fc730160d60 2024-04-26T20:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.990+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7300ceb90 con 0x7fc730160d60 2024-04-26T20:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.991+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71c044020 con 0x7fc730160d60 2024-04-26T20:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.992+0000 7fc72d600640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 0x7fc71006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.992+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc71c0ac220 con 0x7fc730160d60 2024-04-26T20:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.992+0000 7fc735400640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 0x7fc71006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.993+0000 7fc735400640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 0x7fc71006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7300cd400 tx=0x7fc724009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:06.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:06.997+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7300ceb90 con 0x7fc730160d60 2024-04-26T20:01:07.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.230+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc7300693c0 con 0x7fc730160d60 2024-04-26T20:01:07.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.932+0000 7fc72d600640 1 -- 172.21.15.88:0/566075610 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7fc71c032e20 con 0x7fc730160d60 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.936+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 msgr2=0x7fc71006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.936+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 0x7fc71006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7300cd400 tx=0x7fc724009040 comp rx=0 tx=0).stop 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 msgr2=0x7fc7300c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc71c035040 tx=0x7fc71c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 shutdown_connections 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc71006c8f0 0x7fc71006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc730161140 0x7fc730139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc730160d60 0x7fc7300c9f80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 --2- 172.21.15.88:0/566075610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc730146280 0x7fc73015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 >> 172.21.15.88:0/566075610 conn(0x7fc7300b2900 msgr2=0x7fc73006e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 shutdown_connections 2024-04-26T20:01:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:07.937+0000 7fc736800640 1 -- 172.21.15.88:0/566075610 wait complete. 2024-04-26T20:01:07.960 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:01:07.960 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:01:08.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.162+0000 7f59f6600640 1 Processor -- start 2024-04-26T20:01:08.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.162+0000 7f59f6600640 1 -- start start 2024-04-26T20:01:08.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f6600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f59f014cef0 0x7f59f00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f00721b0 0x7f59f0163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f6600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f006e080 con 0x7f59f01640f0 2024-04-26T20:01:08.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f6600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f59f006e360 con 0x7f59f014cef0 2024-04-26T20:01:08.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f59f006e640 con 0x7f59f00721b0 2024-04-26T20:01:08.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f5c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f59f01640f0 0x7f59f00b3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50138/0 (socket says 172.21.15.88:50138) 2024-04-26T20:01:08.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.164+0000 7f59f5200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f59f014cef0 0x7f59f00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:08.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59f5c00640 1 -- 172.21.15.88:0/1706382926 learned_addr learned my addr 172.21.15.88:0/1706382926 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:08.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59efe00640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f00721b0 0x7f59f0163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:08.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59f5200640 1 -- 172.21.15.88:0/1706382926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f00721b0 msgr2=0x7f59f0163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:08.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59f5200640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f00721b0 0x7f59f0163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59f5200640 1 -- 172.21.15.88:0/1706382926 >> v1:172.21.15.88:6789/0 conn(0x7f59f01640f0 legacy=0x7f59f00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.165+0000 7f59f5200640 1 -- 172.21.15.88:0/1706382926 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f014d2d0 con 0x7f59f014cef0 2024-04-26T20:01:08.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.166+0000 7f59f5200640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f014cef0 0x7f59f00730a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59e40097a0 tx=0x7f59e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b99dd00b6ac2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:08.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.166+0000 7f59ef400640 1 -- 172.21.15.88:0/1706382926 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4037070 con 0x7f59f014cef0 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.166+0000 7f59ef400640 1 -- 172.21.15.88:0/1706382926 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4032440 con 0x7f59f014cef0 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.166+0000 7f59ef400640 1 -- 172.21.15.88:0/1706382926 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4032810 con 0x7f59f014cef0 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 -- 172.21.15.88:0/1706382926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f014cef0 msgr2=0x7f59f00730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f014cef0 0x7f59f00730a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f59e40097a0 tx=0x7f59e402a180 comp rx=0 tx=0).stop 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 -- 172.21.15.88:0/1706382926 shutdown_connections 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f00721b0 0x7f59f0163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 --2- 172.21.15.88:0/1706382926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f014cef0 0x7f59f00730a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 -- 172.21.15.88:0/1706382926 >> 172.21.15.88:0/1706382926 conn(0x7f59f00b2900 msgr2=0x7f59f00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 -- 172.21.15.88:0/1706382926 shutdown_connections 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.167+0000 7f59f6600640 1 -- 172.21.15.88:0/1706382926 wait complete. 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.169+0000 7f59f6600640 1 Processor -- start 2024-04-26T20:01:08.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.169+0000 7f59f6600640 1 -- start start 2024-04-26T20:01:08.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59f00721b0 0x7f59f0142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f014cef0 0x7f59f00c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00b4490 con 0x7f59f01640f0 2024-04-26T20:01:08.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f59f00b4310 con 0x7f59f00721b0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f59f006ee40 con 0x7f59f014cef0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53040/0 (socket says 172.21.15.88:53040) 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.170+0000 7f59f5c00640 1 -- 172.21.15.88:0/642304388 learned_addr learned my addr 172.21.15.88:0/642304388 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 -- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f014cef0 msgr2=0x7f59f00c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59efe00640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f014cef0 0x7f59f00c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f014cef0 0x7f59f00c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 -- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59f00721b0 msgr2=0x7f59f0142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59f00721b0 0x7f59f0142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 -- 172.21.15.88:0/642304388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f0144f30 con 0x7f59f01640f0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f5c00640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f59e800caa0 tx=0x7f59e800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:08.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8017070 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e800ab40 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f0145210 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8012400 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.171+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f0145ab0 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.172+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0145600 con 0x7f59f01640f0 2024-04-26T20:01:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.173+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59e801a070 con 0x7f59f01640f0 2024-04-26T20:01:08.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.174+0000 7f59ecc00640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 0x7f59d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:08.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.174+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f59e808b750 con 0x7f59f01640f0 2024-04-26T20:01:08.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.175+0000 7f59f5200640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 0x7f59d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:08.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.175+0000 7f59f5200640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 0x7f59d006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59e4034000 tx=0x7f59e402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:08.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.179+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59e8059490 con 0x7f59f01640f0 2024-04-26T20:01:08.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.419+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f59f00693c0 con 0x7f59f01640f0 2024-04-26T20:01:08.810 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-04-26T20:01:08.809+0000 7f4d03e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:01:08.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.935+0000 7f59ecc00640 1 -- 172.21.15.88:0/642304388 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f59e805d790 con 0x7f59f01640f0 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 msgr2=0x7f59d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 0x7f59d006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f59e4034000 tx=0x7f59e402c000 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 msgr2=0x7f59f0145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f59e800caa0 tx=0x7f59e800cf60 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 shutdown_connections 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59f01640f0 0x7f59f0145fd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f59d006c8f0 0x7f59d006edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f59f014cef0 0x7f59f00c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 --2- 172.21.15.88:0/642304388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59f00721b0 0x7f59f0142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 >> 172.21.15.88:0/642304388 conn(0x7f59f00b2900 msgr2=0x7f59f014c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 shutdown_connections 2024-04-26T20:01:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:08.939+0000 7f59f6600640 1 -- 172.21.15.88:0/642304388 wait complete. 2024-04-26T20:01:08.961 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:01:08.961 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:09.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.162+0000 7fc20b600640 1 Processor -- start 2024-04-26T20:01:09.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.162+0000 7fc20b600640 1 -- start start 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc2041598a0 0x7fc2041638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc20406c770 0x7fc20406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 0x7fc20415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc204159c80 con 0x7fc20406c770 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc20406d110 con 0x7fc2041598a0 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc20b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc204160110 con 0x7fc20415f740 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc203e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc20406c770 0x7fc20406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc203e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc20406c770 0x7fc20406cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58094/0 (socket says 172.21.15.88:58094) 2024-04-26T20:01:09.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.164+0000 7fc203e00640 1 -- 172.21.15.88:0/1203107128 learned_addr learned my addr 172.21.15.88:0/1203107128 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:09.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc208c00640 1 --2- 172.21.15.88:0/1203107128 >> v2:172.21.15.132:3300/0 conn(0x7fc2041598a0 0x7fc2041638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc209600640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 0x7fc20415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc203e00640 1 -- 172.21.15.88:0/1203107128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 msgr2=0x7fc20415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc203e00640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 0x7fc20415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc203e00640 1 -- 172.21.15.88:0/1203107128 >> v2:172.21.15.132:3300/0 conn(0x7fc2041598a0 msgr2=0x7fc2041638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc203e00640 1 --2- 172.21.15.88:0/1203107128 >> v2:172.21.15.132:3300/0 conn(0x7fc2041598a0 0x7fc2041638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc203e00640 1 -- 172.21.15.88:0/1203107128 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040b5fb0 con 0x7fc20406c770 2024-04-26T20:01:09.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.165+0000 7fc209600640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 0x7fc20415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.166+0000 7fc203e00640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20406c770 0x7fc20406cbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2040b0870 tx=0x7fc1ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a138e0beb5ff5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc203400640 1 -- 172.21.15.88:0/1203107128 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec037070 con 0x7fc20406c770 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc203400640 1 -- 172.21.15.88:0/1203107128 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec032440 con 0x7fc20406c770 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc203400640 1 -- 172.21.15.88:0/1203107128 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec032810 con 0x7fc20406c770 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 -- 172.21.15.88:0/1203107128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20406c770 msgr2=0x7fc20406cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20406c770 0x7fc20406cbd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc2040b0870 tx=0x7fc1ec02a180 comp rx=0 tx=0).stop 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 -- 172.21.15.88:0/1203107128 shutdown_connections 2024-04-26T20:01:09.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc20415f740 0x7fc20415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 --2- 172.21.15.88:0/1203107128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20406c770 0x7fc20406cbd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 --2- 172.21.15.88:0/1203107128 >> v2:172.21.15.132:3300/0 conn(0x7fc2041598a0 0x7fc2041638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 -- 172.21.15.88:0/1203107128 >> 172.21.15.88:0/1203107128 conn(0x7fc2040b2d50 msgr2=0x7fc2040b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:09.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 -- 172.21.15.88:0/1203107128 shutdown_connections 2024-04-26T20:01:09.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.167+0000 7fc20b600640 1 -- 172.21.15.88:0/1203107128 wait complete. 2024-04-26T20:01:09.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.169+0000 7fc20b600640 1 Processor -- start 2024-04-26T20:01:09.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.169+0000 7fc20b600640 1 -- start start 2024-04-26T20:01:09.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc20406c770 0x7fc2040ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2041598a0 0x7fc2041436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2041630b0 con 0x7fc20415f740 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc204162dd0 con 0x7fc20406c770 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.170+0000 7fc20b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc204163390 con 0x7fc2041598a0 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58110/0 (socket says 172.21.15.88:58110) 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 -- 172.21.15.88:0/3385229751 learned_addr learned my addr 172.21.15.88:0/3385229751 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc208c00640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc20406c770 0x7fc2040ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc203e00640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2041598a0 0x7fc2041436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 -- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2041598a0 msgr2=0x7fc2041436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2041598a0 0x7fc2041436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 -- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc20406c770 msgr2=0x7fc2040ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc20406c770 0x7fc2040ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 -- 172.21.15.88:0/3385229751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040cea00 con 0x7fc20415f740 2024-04-26T20:01:09.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.171+0000 7fc209600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c7c0 tx=0x7fc1fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.172+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc015020 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.172+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc004230 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.172+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc007620 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.172+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2040cece0 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.172+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2040cf2a0 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.173+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2040b51c0 con 0x7fc20415f740 2024-04-26T20:01:09.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.174+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1fc018020 con 0x7fc20415f740 2024-04-26T20:01:09.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.175+0000 7fc200c00640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 0x7fc1e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.175+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc1fc08bc30 con 0x7fc20415f740 2024-04-26T20:01:09.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.175+0000 7fc208c00640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 0x7fc1e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.176+0000 7fc208c00640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 0x7fc1e006ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009a60 tx=0x7fc1f4003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.180+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2040b51c0 con 0x7fc20415f740 2024-04-26T20:01:09.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.421+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc204069540 con 0x7fc20415f740 2024-04-26T20:01:09.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.422+0000 7fc200c00640 1 -- 172.21.15.88:0/3385229751 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5184 (secure 0 0 0) 0x7fc1fc01f020 con 0x7fc20415f740 2024-04-26T20:01:09.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:09.423 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4705,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2458488289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2458488289},{"type":"v1","addr":"172.21.15.88:6837","nonce":2458488289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:09.423 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 15 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 msgr2=0x7fc1e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 0x7fc1e006ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009a60 tx=0x7fc1f4003990 comp rx=0 tx=0).stop 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 msgr2=0x7fc2040afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c7c0 tx=0x7fc1fc00cc80 comp rx=0 tx=0).stop 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 shutdown_connections 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc20415f740 0x7fc2040afaf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc1e006c920 0x7fc1e006ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2041598a0 0x7fc2041436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 --2- 172.21.15.88:0/3385229751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc20406c770 0x7fc2040ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 >> 172.21.15.88:0/3385229751 conn(0x7fc2040b2d50 msgr2=0x7fc2041611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:09.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 shutdown_connections 2024-04-26T20:01:09.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.425+0000 7fc20b600640 1 -- 172.21.15.88:0/3385229751 wait complete. 2024-04-26T20:01:09.445 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:09.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.644+0000 7f49ff600640 1 Processor -- start 2024-04-26T20:01:09.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.644+0000 7f49ff600640 1 -- start start 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.645+0000 7f49ff600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49f815ef60 0x7f49f815f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.645+0000 7f49ff600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49f80d3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.645+0000 7f49ff600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f80b5d30 con 0x7f49f80d5d90 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.645+0000 7f49ff600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f49f80b6010 con 0x7f49f815ef60 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.645+0000 7f49ff600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f49f80b62f0 con 0x7f49f80d34a0 2024-04-26T20:01:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49f7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f49f80d5d90 0x7f49f80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34330/0 (socket says 172.21.15.88:34330) 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49f7e00640 1 -- 172.21.15.88:0/740406177 learned_addr learned my addr 172.21.15.88:0/740406177 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49fcc00640 1 --2- 172.21.15.88:0/740406177 >> v2:172.21.15.132:3300/0 conn(0x7f49f815ef60 0x7f49f815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49fd600640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49f80d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3399152054 0 0) 0x7f49f80b5d30 con 0x7f49f80d5d90 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.646+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49d8003520 con 0x7f49f80d5d90 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3501670938 0 0) 0x7f49d8003520 con 0x7f49f80d5d90 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 msgr2=0x7f49f80d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49f80d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 >> v2:172.21.15.132:3300/0 conn(0x7f49f815ef60 msgr2=0x7f49f815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 --2- 172.21.15.88:0/740406177 >> v2:172.21.15.132:3300/0 conn(0x7f49f815ef60 0x7f49f815f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f80b65d0 con 0x7f49f80d5d90 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f49e00019f0 con 0x7f49f80d5d90 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 >> v1:172.21.15.88:6789/0 conn(0x7f49f80d5d90 legacy=0x7f49f80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d8003f30 0x7f49d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8004d40 0x7f49d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80b6010 con 0x7f49d8003f30 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80b62f0 con 0x7f49d8004d40 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f49d800b9f0 con 0x7f49f80d34a0 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49f7e00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d8003f30 0x7f49d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49fcc00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.647+0000 7f49fd600640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8004d40 0x7f49d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49f7e00640 1 -- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 msgr2=0x7f49d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49f7e00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49f7e00640 1 -- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8004d40 msgr2=0x7f49d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49f7e00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8004d40 0x7f49d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49f7e00640 1 -- 172.21.15.88:0/740406177 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d8003520 con 0x7f49d8003f30 2024-04-26T20:01:09.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.648+0000 7f49fcc00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80d34a0 0x7f49d80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.649+0000 7f49f7e00640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d8003f30 0x7f49d8004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49e000a880 tx=0x7f49d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1484567ba8f339 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.649+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e000a4a0 con 0x7f49d8003f30 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.649+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e002f070 con 0x7f49d8003f30 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.649+0000 7f49f7400640 1 -- 172.21.15.88:0/740406177 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e002bd40 con 0x7f49d8003f30 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 -- 172.21.15.88:0/740406177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d8003f30 msgr2=0x7f49d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 --2- 172.21.15.88:0/740406177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49d8003f30 0x7f49d8004800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49e000a880 tx=0x7f49d8005830 comp rx=0 tx=0).stop 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49fd600640 1 -- 172.21.15.88:0/740406177 reap_dead start 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 -- 172.21.15.88:0/740406177 shutdown_connections 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 -- 172.21.15.88:0/740406177 >> 172.21.15.88:0/740406177 conn(0x7f49f809e3b0 msgr2=0x7f49f80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 -- 172.21.15.88:0/740406177 shutdown_connections 2024-04-26T20:01:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.650+0000 7f49ff600640 1 -- 172.21.15.88:0/740406177 wait complete. 2024-04-26T20:01:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.651+0000 7f49ff600640 1 Processor -- start 2024-04-26T20:01:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.652+0000 7f49ff600640 1 -- start start 2024-04-26T20:01:09.653 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:01:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8003f30 0x7f49f8161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80b3ad0 0x7f49f814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80d3ab0 con 0x7f49f80b9e60 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80d3930 con 0x7f49d8003f30 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49ff600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f49f80d3c30 con 0x7f49f80b3ad0 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58126/0 (socket says 172.21.15.88:58126) 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 -- 172.21.15.88:0/3040981063 learned_addr learned my addr 172.21.15.88:0/3040981063 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fcc00640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8003f30 0x7f49f8161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49f7e00640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80b3ad0 0x7f49f814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 -- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80b3ad0 msgr2=0x7f49f814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80b3ad0 0x7f49f814bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 -- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8003f30 msgr2=0x7f49f8161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8003f30 0x7f49f8161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.653+0000 7f49fd600640 1 -- 172.21.15.88:0/3040981063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f8161aa0 con 0x7f49f80b9e60 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49fd600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f49f80b7be0 tx=0x7f49f0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0004440 con 0x7f49f80b9e60 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f001b070 con 0x7f49f80b9e60 2024-04-26T20:01:09.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f8161d80 con 0x7f49f80b9e60 2024-04-26T20:01:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0003850 con 0x7f49f80b9e60 2024-04-26T20:01:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.654+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f8162340 con 0x7f49f80b9e60 2024-04-26T20:01:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.655+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f80ba820 con 0x7f49f80b9e60 2024-04-26T20:01:09.660 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:01:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.656+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49f00039f0 con 0x7f49f80b9e60 2024-04-26T20:01:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.658+0000 7f49f4c00640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 0x7f49d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.658+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f49f008bb60 con 0x7f49f80b9e60 2024-04-26T20:01:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.658+0000 7f49fcc00640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 0x7f49d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.659+0000 7f49fcc00640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 0x7f49d406ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009a90 tx=0x7f49e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:09.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.662+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49f0025020 con 0x7f49f80b9e60 2024-04-26T20:01:09.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.931+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49f8069540 con 0x7f49f80b9e60 2024-04-26T20:01:09.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.933+0000 7f49f4c00640 1 -- 172.21.15.88:0/3040981063 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16657 (secure 0 0 0) 0x7f49f0059ef0 con 0x7f49f80b9e60 2024-04-26T20:01:09.935 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:09.935 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":33,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:06.909872+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 msgr2=0x7f49d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 0x7f49d406ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009a90 tx=0x7f49e8008040 comp rx=0 tx=0).stop 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 msgr2=0x7f49f80ba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f49f80b7be0 tx=0x7f49f0011920 comp rx=0 tx=0).stop 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 shutdown_connections 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f80b9e60 0x7f49f80ba2e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f49d406c920 0x7f49d406ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f49f80b3ad0 0x7f49f814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 --2- 172.21.15.88:0/3040981063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49d8003f30 0x7f49f8161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:09.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 >> 172.21.15.88:0/3040981063 conn(0x7f49f809e3b0 msgr2=0x7f49f806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:09.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 shutdown_connections 2024-04-26T20:01:09.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:09.936+0000 7f49ff600640 1 -- 172.21.15.88:0/3040981063 wait complete. 2024-04-26T20:01:09.956 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:01:09.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:01:10.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.156+0000 7f478da00640 1 Processor -- start 2024-04-26T20:01:10.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.156+0000 7f478da00640 1 -- start start 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.157+0000 7f478da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f47880d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.157+0000 7f478da00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4788073ed0 0x7f47880d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.157+0000 7f478da00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4788073110 con 0x7f478815f940 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f478da00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f47880cef40 con 0x7f4788073ed0 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f478da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f47881606b0 con 0x7f4788145570 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4788073ed0 0x7f47880d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4788073ed0 0x7f47880d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58142/0 (socket says 172.21.15.88:58142) 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 -- 172.21.15.88:0/3992969017 learned_addr learned my addr 172.21.15.88:0/3992969017 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f4787e00640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f47880d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 -- 172.21.15.88:0/3992969017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 msgr2=0x7f47880d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f47880d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 -- 172.21.15.88:0/3992969017 >> v1:172.21.15.132:6789/0 conn(0x7f478815f940 legacy=0x7f478815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.158+0000 7f477f400640 1 -- 172.21.15.88:0/3992969017 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47880733f0 con 0x7f4788073ed0 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.160+0000 7f477f400640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4788073ed0 0x7f47880d13b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4770009870 tx=0x7f477002c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d3956ae8a0b516 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.160+0000 7f4787400640 1 -- 172.21.15.88:0/3992969017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770039070 con 0x7f4788073ed0 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.160+0000 7f4787400640 1 -- 172.21.15.88:0/3992969017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4770034640 con 0x7f4788073ed0 2024-04-26T20:01:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.160+0000 7f4787400640 1 -- 172.21.15.88:0/3992969017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770034a10 con 0x7f4788073ed0 2024-04-26T20:01:10.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 -- 172.21.15.88:0/3992969017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4788073ed0 msgr2=0x7f47880d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4788073ed0 0x7f47880d13b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4770009870 tx=0x7f477002c390 comp rx=0 tx=0).stop 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 -- 172.21.15.88:0/3992969017 shutdown_connections 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4788073ed0 0x7f47880d13b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 --2- 172.21.15.88:0/3992969017 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f47880d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 -- 172.21.15.88:0/3992969017 >> 172.21.15.88:0/3992969017 conn(0x7f478806c740 msgr2=0x7f478806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 -- 172.21.15.88:0/3992969017 shutdown_connections 2024-04-26T20:01:10.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.161+0000 7f478da00640 1 -- 172.21.15.88:0/3992969017 wait complete. 2024-04-26T20:01:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.163+0000 7f478da00640 1 Processor -- start 2024-04-26T20:01:10.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.163+0000 7f478da00640 1 -- start start 2024-04-26T20:01:10.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4788073ed0 0x7f478813e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f4788141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4788145950 con 0x7f478815f940 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f47880739b0 con 0x7f4788073ed0 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.164+0000 7f478da00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f47880cdf30 con 0x7f4788145570 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58158/0 (socket says 172.21.15.88:58158) 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 -- 172.21.15.88:0/2873878885 learned_addr learned my addr 172.21.15.88:0/2873878885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f4787e00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4788073ed0 0x7f478813e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f477f400640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f4788141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 -- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 msgr2=0x7f4788141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f4788141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 -- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4788073ed0 msgr2=0x7f478813e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4788073ed0 0x7f478813e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.165+0000 7f478d000640 1 -- 172.21.15.88:0/2873878885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4788140d20 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f478d000640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f478000b9a0 tx=0x7f478000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780019070 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f478000cba0 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4788141000 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780014440 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.166+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4788141ef0 con 0x7f478815f940 2024-04-26T20:01:10.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.167+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47880693c0 con 0x7f478815f940 2024-04-26T20:01:10.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.168+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f478001c070 con 0x7f478815f940 2024-04-26T20:01:10.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.169+0000 7f4784c00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.169+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f478008b770 con 0x7f478815f940 2024-04-26T20:01:10.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.169+0000 7f4787e00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.170+0000 7f4787e00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 0x7f476406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f477800a970 tx=0x7f4778008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.173+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4780059ba0 con 0x7f478815f940 2024-04-26T20:01:10.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.361+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47880729e0 con 0x7f476406c8f0 2024-04-26T20:01:10.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.368+0000 7f4784c00640 1 -- 172.21.15.88:0/2873878885 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246631 (secure 0 0 0) 0x7f47880729e0 con 0x7f476406c8f0 2024-04-26T20:01:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:10.374 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:01:10.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.374+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 msgr2=0x7f476406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.374+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 0x7f476406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f477800a970 tx=0x7f4778008040 comp rx=0 tx=0).stop 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.374+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 msgr2=0x7f47881419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.374+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f478000b9a0 tx=0x7f478000be60 comp rx=0 tx=0).stop 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 shutdown_connections 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f478815f940 0x7f47881419b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4788145570 0x7f4788141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 --2- 172.21.15.88:0/2873878885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4788073ed0 0x7f478813e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 >> 172.21.15.88:0/2873878885 conn(0x7f478806c740 msgr2=0x7f47880ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:10.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 shutdown_connections 2024-04-26T20:01:10.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.375+0000 7f478da00640 1 -- 172.21.15.88:0/2873878885 wait complete. 2024-04-26T20:01:10.394 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T20:01:08.965073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219956,"kb_used_data":3784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754764,"statfs":{"total":773094113280,"available":772868878336,"internally_reserved":0,"allocated":3874816,"data_stored":2200610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.106416"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886701+0000","last_change":"2024-04-26T20:01:04.886701+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886701+0000","last_undegraded":"2024-04-26T20:01:04.886701+0000","last_fullsized":"2024-04-26T20:01:04.886701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.896075+0000","last_change":"2024-04-26T20:01:04.896075+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.896075+0000","last_undegraded":"2024-04-26T20:01:04.896075+0000","last_fullsized":"2024-04-26T20:01:04.896075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880267+0000","last_change":"2024-04-26T20:01:04.880267+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880267+0000","last_undegraded":"2024-04-26T20:01:04.880267+0000","last_fullsized":"2024-04-26T20:01:04.880267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880742+0000","last_change":"2024-04-26T20:01:04.880742+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880742+0000","last_undegraded":"2024-04-26T20:01:04.880742+0000","last_fullsized":"2024-04-26T20:01:04.880742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872740+0000","last_change":"2024-04-26T20:01:04.872740+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872740+0000","last_undegraded":"2024-04-26T20:01:04.872740+0000","last_fullsized":"2024-04-26T20:01:04.872740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884638+0000","last_change":"2024-04-26T20:01:04.884638+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884638+0000","last_undegraded":"2024-04-26T20:01:04.884638+0000","last_fullsized":"2024-04-26T20:01:04.884638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889717+0000","last_change":"2024-04-26T20:01:04.889717+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889717+0000","last_undegraded":"2024-04-26T20:01:04.889717+0000","last_fullsized":"2024-04-26T20:01:04.889717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.895162+0000","last_change":"2024-04-26T20:01:04.895162+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.895162+0000","last_undegraded":"2024-04-26T20:01:04.895162+0000","last_fullsized":"2024-04-26T20:01:04.895162+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876488+0000","last_change":"2024-04-26T20:01:04.876488+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876488+0000","last_undegraded":"2024-04-26T20:01:04.876488+0000","last_fullsized":"2024-04-26T20:01:04.876488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878397+0000","last_change":"2024-04-26T20:01:04.878397+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878397+0000","last_undegraded":"2024-04-26T20:01:04.878397+0000","last_fullsized":"2024-04-26T20:01:04.878397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.892827+0000","last_change":"2024-04-26T20:01:04.892827+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.892827+0000","last_undegraded":"2024-04-26T20:01:04.892827+0000","last_fullsized":"2024-04-26T20:01:04.892827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.881971+0000","last_change":"2024-04-26T20:01:04.881971+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.881971+0000","last_undegraded":"2024-04-26T20:01:04.881971+0000","last_fullsized":"2024-04-26T20:01:04.881971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871892+0000","last_change":"2024-04-26T20:01:04.871892+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871892+0000","last_undegraded":"2024-04-26T20:01:04.871892+0000","last_fullsized":"2024-04-26T20:01:04.871892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884048+0000","last_change":"2024-04-26T20:01:04.884048+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884048+0000","last_undegraded":"2024-04-26T20:01:04.884048+0000","last_fullsized":"2024-04-26T20:01:04.884048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884458+0000","last_change":"2024-04-26T20:01:04.884458+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884458+0000","last_undegraded":"2024-04-26T20:01:04.884458+0000","last_fullsized":"2024-04-26T20:01:04.884458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.891186+0000","last_change":"2024-04-26T20:01:04.891186+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.891186+0000","last_undegraded":"2024-04-26T20:01:04.891186+0000","last_fullsized":"2024-04-26T20:01:04.891186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.893989+0000","last_change":"2024-04-26T20:01:04.893989+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.893989+0000","last_undegraded":"2024-04-26T20:01:04.893989+0000","last_fullsized":"2024-04-26T20:01:04.893989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889725+0000","last_change":"2024-04-26T20:01:04.889725+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889725+0000","last_undegraded":"2024-04-26T20:01:04.889725+0000","last_fullsized":"2024-04-26T20:01:04.889725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888816+0000","last_change":"2024-04-26T20:01:04.888816+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888816+0000","last_undegraded":"2024-04-26T20:01:04.888816+0000","last_fullsized":"2024-04-26T20:01:04.888816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871091+0000","last_change":"2024-04-26T20:01:04.871091+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871091+0000","last_undegraded":"2024-04-26T20:01:04.871091+0000","last_fullsized":"2024-04-26T20:01:04.871091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878594+0000","last_change":"2024-04-26T20:01:04.878594+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878594+0000","last_undegraded":"2024-04-26T20:01:04.878594+0000","last_fullsized":"2024-04-26T20:01:04.878594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.874045+0000","last_change":"2024-04-26T20:01:04.874045+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.874045+0000","last_undegraded":"2024-04-26T20:01:04.874045+0000","last_fullsized":"2024-04-26T20:01:04.874045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.879603+0000","last_change":"2024-04-26T20:01:04.879603+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.879603+0000","last_undegraded":"2024-04-26T20:01:04.879603+0000","last_fullsized":"2024-04-26T20:01:04.879603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880789+0000","last_change":"2024-04-26T20:01:04.880789+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880789+0000","last_undegraded":"2024-04-26T20:01:04.880789+0000","last_fullsized":"2024-04-26T20:01:04.880789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878875+0000","last_change":"2024-04-26T20:01:04.878875+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878875+0000","last_undegraded":"2024-04-26T20:01:04.878875+0000","last_fullsized":"2024-04-26T20:01:04.878875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886270+0000","last_change":"2024-04-26T20:01:04.886270+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886270+0000","last_undegraded":"2024-04-26T20:01:04.886270+0000","last_fullsized":"2024-04-26T20:01:04.886270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878653+0000","last_change":"2024-04-26T20:01:04.878653+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878653+0000","last_undegraded":"2024-04-26T20:01:04.878653+0000","last_fullsized":"2024-04-26T20:01:04.878653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880229+0000","last_change":"2024-04-26T20:01:04.880229+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880229+0000","last_undegraded":"2024-04-26T20:01:04.880229+0000","last_fullsized":"2024-04-26T20:01:04.880229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876370+0000","last_change":"2024-04-26T20:01:04.876370+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876370+0000","last_undegraded":"2024-04-26T20:01:04.876370+0000","last_fullsized":"2024-04-26T20:01:04.876370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889894+0000","last_change":"2024-04-26T20:01:04.889894+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889894+0000","last_undegraded":"2024-04-26T20:01:04.889894+0000","last_fullsized":"2024-04-26T20:01:04.889894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888045+0000","last_change":"2024-04-26T20:01:04.888045+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888045+0000","last_undegraded":"2024-04-26T20:01:04.888045+0000","last_fullsized":"2024-04-26T20:01:04.888045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.894702+0000","last_change":"2024-04-26T20:01:04.894702+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.894702+0000","last_undegraded":"2024-04-26T20:01:04.894702+0000","last_fullsized":"2024-04-26T20:01:04.894702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.879971+0000","last_change":"2024-04-26T20:01:04.879971+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.879971+0000","last_undegraded":"2024-04-26T20:01:04.879971+0000","last_fullsized":"2024-04-26T20:01:04.879971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905028+0000","last_change":"2024-04-26T20:01:04.900074+0000","last_active":"2024-04-26T20:01:04.905028+0000","last_peered":"2024-04-26T20:01:04.905028+0000","last_clean":"2024-04-26T20:01:04.905028+0000","last_became_active":"2024-04-26T20:01:04.898252+0000","last_became_peered":"2024-04-26T20:01:04.898252+0000","last_unstale":"2024-04-26T20:01:04.905028+0000","last_undegraded":"2024-04-26T20:01:04.905028+0000","last_fullsized":"2024-04-26T20:01:04.905028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885982+0000","last_change":"2024-04-26T20:01:04.885982+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885982+0000","last_undegraded":"2024-04-26T20:01:04.885982+0000","last_fullsized":"2024-04-26T20:01:04.885982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904527+0000","last_change":"2024-04-26T20:01:04.902720+0000","last_active":"2024-04-26T20:01:04.904527+0000","last_peered":"2024-04-26T20:01:04.904527+0000","last_clean":"2024-04-26T20:01:04.904527+0000","last_became_active":"2024-04-26T20:01:04.902158+0000","last_became_peered":"2024-04-26T20:01:04.902158+0000","last_unstale":"2024-04-26T20:01:04.904527+0000","last_undegraded":"2024-04-26T20:01:04.904527+0000","last_fullsized":"2024-04-26T20:01:04.904527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.874062+0000","last_change":"2024-04-26T20:01:04.874062+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.874062+0000","last_undegraded":"2024-04-26T20:01:04.874062+0000","last_fullsized":"2024-04-26T20:01:04.874062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.901408+0000","last_change":"2024-04-26T20:01:04.897236+0000","last_active":"2024-04-26T20:01:04.901408+0000","last_peered":"2024-04-26T20:01:04.901408+0000","last_clean":"2024-04-26T20:01:04.901408+0000","last_became_active":"2024-04-26T20:01:04.896622+0000","last_became_peered":"2024-04-26T20:01:04.896622+0000","last_unstale":"2024-04-26T20:01:04.901408+0000","last_undegraded":"2024-04-26T20:01:04.901408+0000","last_fullsized":"2024-04-26T20:01:04.901408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.877692+0000","last_change":"2024-04-26T20:01:04.877692+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.877692+0000","last_undegraded":"2024-04-26T20:01:04.877692+0000","last_fullsized":"2024-04-26T20:01:04.877692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.884619+0000","last_change":"2024-04-26T20:01:04.879892+0000","last_active":"2024-04-26T20:01:04.884619+0000","last_peered":"2024-04-26T20:01:04.884619+0000","last_clean":"2024-04-26T20:01:04.884619+0000","last_became_active":"2024-04-26T20:01:04.879459+0000","last_became_peered":"2024-04-26T20:01:04.879459+0000","last_unstale":"2024-04-26T20:01:04.884619+0000","last_undegraded":"2024-04-26T20:01:04.884619+0000","last_fullsized":"2024-04-26T20:01:04.884619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:58:34.027761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889370+0000","last_change":"2024-04-26T20:01:04.889370+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889370+0000","last_undegraded":"2024-04-26T20:01:04.889370+0000","last_fullsized":"2024-04-26T20:01:04.889370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905145+0000","last_change":"2024-04-26T20:01:04.899358+0000","last_active":"2024-04-26T20:01:04.905145+0000","last_peered":"2024-04-26T20:01:04.905145+0000","last_clean":"2024-04-26T20:01:04.905145+0000","last_became_active":"2024-04-26T20:01:04.899066+0000","last_became_peered":"2024-04-26T20:01:04.899066+0000","last_unstale":"2024-04-26T20:01:04.905145+0000","last_undegraded":"2024-04-26T20:01:04.905145+0000","last_fullsized":"2024-04-26T20:01:04.905145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:32.524975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880387+0000","last_change":"2024-04-26T20:01:04.880387+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880387+0000","last_undegraded":"2024-04-26T20:01:04.880387+0000","last_fullsized":"2024-04-26T20:01:04.880387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.903001+0000","last_change":"2024-04-26T20:01:04.896700+0000","last_active":"2024-04-26T20:01:04.903001+0000","last_peered":"2024-04-26T20:01:04.903001+0000","last_clean":"2024-04-26T20:01:04.903001+0000","last_became_active":"2024-04-26T20:01:04.896384+0000","last_became_peered":"2024-04-26T20:01:04.896384+0000","last_unstale":"2024-04-26T20:01:04.903001+0000","last_undegraded":"2024-04-26T20:01:04.903001+0000","last_fullsized":"2024-04-26T20:01:04.903001+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:13.813164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872903+0000","last_change":"2024-04-26T20:01:04.872903+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872903+0000","last_undegraded":"2024-04-26T20:01:04.872903+0000","last_fullsized":"2024-04-26T20:01:04.872903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.894812+0000","last_change":"2024-04-26T20:01:04.891168+0000","last_active":"2024-04-26T20:01:04.894812+0000","last_peered":"2024-04-26T20:01:04.894812+0000","last_clean":"2024-04-26T20:01:04.894812+0000","last_became_active":"2024-04-26T20:01:04.890897+0000","last_became_peered":"2024-04-26T20:01:04.890897+0000","last_unstale":"2024-04-26T20:01:04.894812+0000","last_undegraded":"2024-04-26T20:01:04.894812+0000","last_fullsized":"2024-04-26T20:01:04.894812+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876486+0000","last_change":"2024-04-26T20:01:04.876486+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876486+0000","last_undegraded":"2024-04-26T20:01:04.876486+0000","last_fullsized":"2024-04-26T20:01:04.876486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904980+0000","last_change":"2024-04-26T20:01:04.900028+0000","last_active":"2024-04-26T20:01:04.904980+0000","last_peered":"2024-04-26T20:01:04.904980+0000","last_clean":"2024-04-26T20:01:04.904980+0000","last_became_active":"2024-04-26T20:01:04.899740+0000","last_became_peered":"2024-04-26T20:01:04.899740+0000","last_unstale":"2024-04-26T20:01:04.904980+0000","last_undegraded":"2024-04-26T20:01:04.904980+0000","last_fullsized":"2024-04-26T20:01:04.904980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:19.885170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880346+0000","last_change":"2024-04-26T20:01:04.880346+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880346+0000","last_undegraded":"2024-04-26T20:01:04.880346+0000","last_fullsized":"2024-04-26T20:01:04.880346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905563+0000","last_change":"2024-04-26T20:01:04.904429+0000","last_active":"2024-04-26T20:01:04.905563+0000","last_peered":"2024-04-26T20:01:04.905563+0000","last_clean":"2024-04-26T20:01:04.905563+0000","last_became_active":"2024-04-26T20:01:04.904127+0000","last_became_peered":"2024-04-26T20:01:04.904127+0000","last_unstale":"2024-04-26T20:01:04.905563+0000","last_undegraded":"2024-04-26T20:01:04.905563+0000","last_fullsized":"2024-04-26T20:01:04.905563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885324+0000","last_change":"2024-04-26T20:01:04.885324+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885324+0000","last_undegraded":"2024-04-26T20:01:04.885324+0000","last_fullsized":"2024-04-26T20:01:04.885324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904502+0000","last_change":"2024-04-26T20:01:04.902692+0000","last_active":"2024-04-26T20:01:04.904502+0000","last_peered":"2024-04-26T20:01:04.904502+0000","last_clean":"2024-04-26T20:01:04.904502+0000","last_became_active":"2024-04-26T20:01:04.901817+0000","last_became_peered":"2024-04-26T20:01:04.901817+0000","last_unstale":"2024-04-26T20:01:04.904502+0000","last_undegraded":"2024-04-26T20:01:04.904502+0000","last_fullsized":"2024-04-26T20:01:04.904502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889418+0000","last_change":"2024-04-26T20:01:04.889418+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889418+0000","last_undegraded":"2024-04-26T20:01:04.889418+0000","last_fullsized":"2024-04-26T20:01:04.889418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905160+0000","last_change":"2024-04-26T20:01:04.903315+0000","last_active":"2024-04-26T20:01:04.905160+0000","last_peered":"2024-04-26T20:01:04.905160+0000","last_clean":"2024-04-26T20:01:04.905160+0000","last_became_active":"2024-04-26T20:01:04.902750+0000","last_became_peered":"2024-04-26T20:01:04.902750+0000","last_unstale":"2024-04-26T20:01:04.905160+0000","last_undegraded":"2024-04-26T20:01:04.905160+0000","last_fullsized":"2024-04-26T20:01:04.905160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.883743+0000","last_change":"2024-04-26T20:01:04.883743+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.883743+0000","last_undegraded":"2024-04-26T20:01:04.883743+0000","last_fullsized":"2024-04-26T20:01:04.883743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900539+0000","last_change":"2024-04-26T20:01:04.896296+0000","last_active":"2024-04-26T20:01:04.900539+0000","last_peered":"2024-04-26T20:01:04.900539+0000","last_clean":"2024-04-26T20:01:04.900539+0000","last_became_active":"2024-04-26T20:01:04.896034+0000","last_became_peered":"2024-04-26T20:01:04.896034+0000","last_unstale":"2024-04-26T20:01:04.900539+0000","last_undegraded":"2024-04-26T20:01:04.900539+0000","last_fullsized":"2024-04-26T20:01:04.900539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876389+0000","last_change":"2024-04-26T20:01:04.876389+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876389+0000","last_undegraded":"2024-04-26T20:01:04.876389+0000","last_fullsized":"2024-04-26T20:01:04.876389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905952+0000","last_change":"2024-04-26T20:01:04.902304+0000","last_active":"2024-04-26T20:01:04.905952+0000","last_peered":"2024-04-26T20:01:04.905952+0000","last_clean":"2024-04-26T20:01:04.905952+0000","last_became_active":"2024-04-26T20:01:04.901986+0000","last_became_peered":"2024-04-26T20:01:04.901986+0000","last_unstale":"2024-04-26T20:01:04.905952+0000","last_undegraded":"2024-04-26T20:01:04.905952+0000","last_fullsized":"2024-04-26T20:01:04.905952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:33.368463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.890900+0000","last_change":"2024-04-26T20:01:04.890900+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.890900+0000","last_undegraded":"2024-04-26T20:01:04.890900+0000","last_fullsized":"2024-04-26T20:01:04.890900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900398+0000","last_change":"2024-04-26T20:01:04.898231+0000","last_active":"2024-04-26T20:01:04.900398+0000","last_peered":"2024-04-26T20:01:04.900398+0000","last_clean":"2024-04-26T20:01:04.900398+0000","last_became_active":"2024-04-26T20:01:04.896868+0000","last_became_peered":"2024-04-26T20:01:04.896868+0000","last_unstale":"2024-04-26T20:01:04.900398+0000","last_undegraded":"2024-04-26T20:01:04.900398+0000","last_fullsized":"2024-04-26T20:01:04.900398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.892352+0000","last_change":"2024-04-26T20:01:04.892352+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.892352+0000","last_undegraded":"2024-04-26T20:01:04.892352+0000","last_fullsized":"2024-04-26T20:01:04.892352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905254+0000","last_change":"2024-04-26T20:01:04.904256+0000","last_active":"2024-04-26T20:01:04.905254+0000","last_peered":"2024-04-26T20:01:04.905254+0000","last_clean":"2024-04-26T20:01:04.905254+0000","last_became_active":"2024-04-26T20:01:04.903998+0000","last_became_peered":"2024-04-26T20:01:04.903998+0000","last_unstale":"2024-04-26T20:01:04.905254+0000","last_undegraded":"2024-04-26T20:01:04.905254+0000","last_fullsized":"2024-04-26T20:01:04.905254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886188+0000","last_change":"2024-04-26T20:01:04.886188+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886188+0000","last_undegraded":"2024-04-26T20:01:04.886188+0000","last_fullsized":"2024-04-26T20:01:04.886188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897458+0000","last_change":"2024-04-26T20:01:04.894197+0000","last_active":"2024-04-26T20:01:04.897458+0000","last_peered":"2024-04-26T20:01:04.897458+0000","last_clean":"2024-04-26T20:01:04.897458+0000","last_became_active":"2024-04-26T20:01:04.893919+0000","last_became_peered":"2024-04-26T20:01:04.893919+0000","last_unstale":"2024-04-26T20:01:04.897458+0000","last_undegraded":"2024-04-26T20:01:04.897458+0000","last_fullsized":"2024-04-26T20:01:04.897458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876928+0000","last_change":"2024-04-26T20:01:04.876928+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876928+0000","last_undegraded":"2024-04-26T20:01:04.876928+0000","last_fullsized":"2024-04-26T20:01:04.876928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900644+0000","last_change":"2024-04-26T20:01:04.898895+0000","last_active":"2024-04-26T20:01:04.900644+0000","last_peered":"2024-04-26T20:01:04.900644+0000","last_clean":"2024-04-26T20:01:04.900644+0000","last_became_active":"2024-04-26T20:01:04.898535+0000","last_became_peered":"2024-04-26T20:01:04.898535+0000","last_unstale":"2024-04-26T20:01:04.900644+0000","last_undegraded":"2024-04-26T20:01:04.900644+0000","last_fullsized":"2024-04-26T20:01:04.900644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885655+0000","last_change":"2024-04-26T20:01:04.885655+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885655+0000","last_undegraded":"2024-04-26T20:01:04.885655+0000","last_fullsized":"2024-04-26T20:01:04.885655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905993+0000","last_change":"2024-04-26T20:01:04.902861+0000","last_active":"2024-04-26T20:01:04.905993+0000","last_peered":"2024-04-26T20:01:04.905993+0000","last_clean":"2024-04-26T20:01:04.905993+0000","last_became_active":"2024-04-26T20:01:04.902393+0000","last_became_peered":"2024-04-26T20:01:04.902393+0000","last_unstale":"2024-04-26T20:01:04.905993+0000","last_undegraded":"2024-04-26T20:01:04.905993+0000","last_fullsized":"2024-04-26T20:01:04.905993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876312+0000","last_change":"2024-04-26T20:01:04.876312+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876312+0000","last_undegraded":"2024-04-26T20:01:04.876312+0000","last_fullsized":"2024-04-26T20:01:04.876312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897922+0000","last_change":"2024-04-26T20:01:04.887577+0000","last_active":"2024-04-26T20:01:04.897922+0000","last_peered":"2024-04-26T20:01:04.897922+0000","last_clean":"2024-04-26T20:01:04.897922+0000","last_became_active":"2024-04-26T20:01:04.887059+0000","last_became_peered":"2024-04-26T20:01:04.887059+0000","last_unstale":"2024-04-26T20:01:04.897922+0000","last_undegraded":"2024-04-26T20:01:04.897922+0000","last_fullsized":"2024-04-26T20:01:04.897922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:13.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872448+0000","last_change":"2024-04-26T20:01:04.872448+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872448+0000","last_undegraded":"2024-04-26T20:01:04.872448+0000","last_fullsized":"2024-04-26T20:01:04.872448+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904682+0000","last_change":"2024-04-26T20:01:04.897364+0000","last_active":"2024-04-26T20:01:04.904682+0000","last_peered":"2024-04-26T20:01:04.904682+0000","last_clean":"2024-04-26T20:01:04.904682+0000","last_became_active":"2024-04-26T20:01:04.896186+0000","last_became_peered":"2024-04-26T20:01:04.896186+0000","last_unstale":"2024-04-26T20:01:04.904682+0000","last_undegraded":"2024-04-26T20:01:04.904682+0000","last_fullsized":"2024-04-26T20:01:04.904682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:58:34.027761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.870312+0000","last_change":"2024-04-26T20:01:04.870312+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.870312+0000","last_undegraded":"2024-04-26T20:01:04.870312+0000","last_fullsized":"2024-04-26T20:01:04.870312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.891773+0000","last_change":"2024-04-26T20:01:04.889881+0000","last_active":"2024-04-26T20:01:04.891773+0000","last_peered":"2024-04-26T20:01:04.891773+0000","last_clean":"2024-04-26T20:01:04.891773+0000","last_became_active":"2024-04-26T20:01:04.889623+0000","last_became_peered":"2024-04-26T20:01:04.889623+0000","last_unstale":"2024-04-26T20:01:04.891773+0000","last_undegraded":"2024-04-26T20:01:04.891773+0000","last_fullsized":"2024-04-26T20:01:04.891773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871118+0000","last_change":"2024-04-26T20:01:04.871118+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871118+0000","last_undegraded":"2024-04-26T20:01:04.871118+0000","last_fullsized":"2024-04-26T20:01:04.871118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905070+0000","last_change":"2024-04-26T20:01:04.901055+0000","last_active":"2024-04-26T20:01:04.905070+0000","last_peered":"2024-04-26T20:01:04.905070+0000","last_clean":"2024-04-26T20:01:04.905070+0000","last_became_active":"2024-04-26T20:01:04.900083+0000","last_became_peered":"2024-04-26T20:01:04.900083+0000","last_unstale":"2024-04-26T20:01:04.905070+0000","last_undegraded":"2024-04-26T20:01:04.905070+0000","last_fullsized":"2024-04-26T20:01:04.905070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:19.030356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872842+0000","last_change":"2024-04-26T20:01:04.872842+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872842+0000","last_undegraded":"2024-04-26T20:01:04.872842+0000","last_fullsized":"2024-04-26T20:01:04.872842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897983+0000","last_change":"2024-04-26T20:01:04.888976+0000","last_active":"2024-04-26T20:01:04.897983+0000","last_peered":"2024-04-26T20:01:04.897983+0000","last_clean":"2024-04-26T20:01:04.897983+0000","last_became_active":"2024-04-26T20:01:04.888367+0000","last_became_peered":"2024-04-26T20:01:04.888367+0000","last_unstale":"2024-04-26T20:01:04.897983+0000","last_undegraded":"2024-04-26T20:01:04.897983+0000","last_fullsized":"2024-04-26T20:01:04.897983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.864035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.890176+0000","last_change":"2024-04-26T20:01:04.890176+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.890176+0000","last_undegraded":"2024-04-26T20:01:04.890176+0000","last_fullsized":"2024-04-26T20:01:04.890176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.910725+0000","last_change":"2024-04-26T20:01:04.906119+0000","last_active":"2024-04-26T20:01:04.910725+0000","last_peered":"2024-04-26T20:01:04.910725+0000","last_clean":"2024-04-26T20:01:04.910725+0000","last_became_active":"2024-04-26T20:01:04.905793+0000","last_became_peered":"2024-04-26T20:01:04.905793+0000","last_unstale":"2024-04-26T20:01:04.910725+0000","last_undegraded":"2024-04-26T20:01:04.910725+0000","last_fullsized":"2024-04-26T20:01:04.910725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.870789+0000","last_change":"2024-04-26T20:01:04.870789+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.870789+0000","last_undegraded":"2024-04-26T20:01:04.870789+0000","last_fullsized":"2024-04-26T20:01:04.870789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900429+0000","last_change":"2024-04-26T20:01:04.897905+0000","last_active":"2024-04-26T20:01:04.900429+0000","last_peered":"2024-04-26T20:01:04.900429+0000","last_clean":"2024-04-26T20:01:04.900429+0000","last_became_active":"2024-04-26T20:01:04.897556+0000","last_became_peered":"2024-04-26T20:01:04.897556+0000","last_unstale":"2024-04-26T20:01:04.900429+0000","last_undegraded":"2024-04-26T20:01:04.900429+0000","last_fullsized":"2024-04-26T20:01:04.900429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884382+0000","last_change":"2024-04-26T20:01:04.884382+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884382+0000","last_undegraded":"2024-04-26T20:01:04.884382+0000","last_fullsized":"2024-04-26T20:01:04.884382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.890435+0000","last_change":"2024-04-26T20:01:04.888031+0000","last_active":"2024-04-26T20:01:04.890435+0000","last_peered":"2024-04-26T20:01:04.890435+0000","last_clean":"2024-04-26T20:01:04.890435+0000","last_became_active":"2024-04-26T20:01:04.887522+0000","last_became_peered":"2024-04-26T20:01:04.887522+0000","last_unstale":"2024-04-26T20:01:04.890435+0000","last_undegraded":"2024-04-26T20:01:04.890435+0000","last_fullsized":"2024-04-26T20:01:04.890435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880299+0000","last_change":"2024-04-26T20:01:04.880299+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880299+0000","last_undegraded":"2024-04-26T20:01:04.880299+0000","last_fullsized":"2024-04-26T20:01:04.880299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.887119+0000","last_change":"2024-04-26T20:01:04.882527+0000","last_active":"2024-04-26T20:01:04.887119+0000","last_peered":"2024-04-26T20:01:04.887119+0000","last_clean":"2024-04-26T20:01:04.887119+0000","last_became_active":"2024-04-26T20:01:04.881905+0000","last_became_peered":"2024-04-26T20:01:04.881905+0000","last_unstale":"2024-04-26T20:01:04.887119+0000","last_undegraded":"2024-04-26T20:01:04.887119+0000","last_fullsized":"2024-04-26T20:01:04.887119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888846+0000","last_change":"2024-04-26T20:01:04.888846+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888846+0000","last_undegraded":"2024-04-26T20:01:04.888846+0000","last_fullsized":"2024-04-26T20:01:04.888846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900718+0000","last_change":"2024-04-26T20:01:04.898199+0000","last_active":"2024-04-26T20:01:04.900718+0000","last_peered":"2024-04-26T20:01:04.900718+0000","last_clean":"2024-04-26T20:01:04.900718+0000","last_became_active":"2024-04-26T20:01:04.897910+0000","last_became_peered":"2024-04-26T20:01:04.897910+0000","last_unstale":"2024-04-26T20:01:04.900718+0000","last_undegraded":"2024-04-26T20:01:04.900718+0000","last_fullsized":"2024-04-26T20:01:04.900718+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880280+0000","last_change":"2024-04-26T20:01:04.880280+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880280+0000","last_undegraded":"2024-04-26T20:01:04.880280+0000","last_fullsized":"2024-04-26T20:01:04.880280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900425+0000","last_change":"2024-04-26T20:01:04.898465+0000","last_active":"2024-04-26T20:01:04.900425+0000","last_peered":"2024-04-26T20:01:04.900425+0000","last_clean":"2024-04-26T20:01:04.900425+0000","last_became_active":"2024-04-26T20:01:04.897963+0000","last_became_peered":"2024-04-26T20:01:04.897963+0000","last_unstale":"2024-04-26T20:01:04.900425+0000","last_undegraded":"2024-04-26T20:01:04.900425+0000","last_fullsized":"2024-04-26T20:01:04.900425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880393+0000","last_change":"2024-04-26T20:01:04.880393+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880393+0000","last_undegraded":"2024-04-26T20:01:04.880393+0000","last_fullsized":"2024-04-26T20:01:04.880393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.901497+0000","last_change":"2024-04-26T20:01:04.894822+0000","last_active":"2024-04-26T20:01:04.901497+0000","last_peered":"2024-04-26T20:01:04.901497+0000","last_clean":"2024-04-26T20:01:04.901497+0000","last_became_active":"2024-04-26T20:01:04.893421+0000","last_became_peered":"2024-04-26T20:01:04.893421+0000","last_unstale":"2024-04-26T20:01:04.901497+0000","last_undegraded":"2024-04-26T20:01:04.901497+0000","last_fullsized":"2024-04-26T20:01:04.901497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:42.345562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.877406+0000","last_change":"2024-04-26T20:01:04.877406+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.877406+0000","last_undegraded":"2024-04-26T20:01:04.877406+0000","last_fullsized":"2024-04-26T20:01:04.877406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905190+0000","last_change":"2024-04-26T20:01:04.904249+0000","last_active":"2024-04-26T20:01:04.905190+0000","last_peered":"2024-04-26T20:01:04.905190+0000","last_clean":"2024-04-26T20:01:04.905190+0000","last_became_active":"2024-04-26T20:01:04.903995+0000","last_became_peered":"2024-04-26T20:01:04.903995+0000","last_unstale":"2024-04-26T20:01:04.905190+0000","last_undegraded":"2024-04-26T20:01:04.905190+0000","last_fullsized":"2024-04-26T20:01:04.905190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889392+0000","last_change":"2024-04-26T20:01:04.889392+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889392+0000","last_undegraded":"2024-04-26T20:01:04.889392+0000","last_fullsized":"2024-04-26T20:01:04.889392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905888+0000","last_change":"2024-04-26T20:01:04.903742+0000","last_active":"2024-04-26T20:01:04.905888+0000","last_peered":"2024-04-26T20:01:04.905888+0000","last_clean":"2024-04-26T20:01:04.905888+0000","last_became_active":"2024-04-26T20:01:04.903299+0000","last_became_peered":"2024-04-26T20:01:04.903299+0000","last_unstale":"2024-04-26T20:01:04.905888+0000","last_undegraded":"2024-04-26T20:01:04.905888+0000","last_fullsized":"2024-04-26T20:01:04.905888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:07.281520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.899769+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:04.899769+0000","last_peered":"2024-04-26T20:01:04.899769+0000","last_clean":"2024-04-26T20:01:04.899769+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:04.899769+0000","last_undegraded":"2024-04-26T20:01:04.899769+0000","last_fullsized":"2024-04-26T20:01:04.899769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":711979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:01:10.398 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:01:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.637+0000 7fbcb4000640 1 Processor -- start 2024-04-26T20:01:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.637+0000 7fbcb4000640 1 -- start start 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac15cdf0 0x7fbcac0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb4000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbcac0cd650 0x7fbcac06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb4000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcac0af9a0 con 0x7fbcac0b4f50 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb4000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fbcac0ae2c0 con 0x7fbcac0cd650 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbcac0b05b0 con 0x7fbcac15cdf0 2024-04-26T20:01:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb0c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbcac0cd650 0x7fbcac06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb0c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbcac0cd650 0x7fbcac06c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58168/0 (socket says 172.21.15.88:58168) 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.639+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2105102827 learned_addr learned my addr 172.21.15.88:0/2105102827 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.640+0000 7fbcb1600640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac15cdf0 0x7fbcac0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.640+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2105102827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac15cdf0 msgr2=0x7fbcac0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.640+0000 7fbcb0c00640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac15cdf0 0x7fbcac0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.640+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2105102827 >> v1:172.21.15.132:6789/0 conn(0x7fbcac0b4f50 legacy=0x7fbcac072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.640+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2105102827 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac0afc80 con 0x7fbcac0cd650 2024-04-26T20:01:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.641+0000 7fbcb0c00640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac06c770 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009870 tx=0x7fbc9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87135f23f54b4281 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcabe00640 1 -- 172.21.15.88:0/2105102827 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98037070 con 0x7fbcac0cd650 2024-04-26T20:01:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcabe00640 1 -- 172.21.15.88:0/2105102827 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc98032440 con 0x7fbcac0cd650 2024-04-26T20:01:10.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcabe00640 1 -- 172.21.15.88:0/2105102827 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98032810 con 0x7fbcac0cd650 2024-04-26T20:01:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 -- 172.21.15.88:0/2105102827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 msgr2=0x7fbcac06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac06c770 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009870 tx=0x7fbc9802a210 comp rx=0 tx=0).stop 2024-04-26T20:01:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 -- 172.21.15.88:0/2105102827 shutdown_connections 2024-04-26T20:01:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac06c770 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2105102827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac15cdf0 0x7fbcac0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.642+0000 7fbcb4000640 1 -- 172.21.15.88:0/2105102827 >> 172.21.15.88:0/2105102827 conn(0x7fbcac0b2d50 msgr2=0x7fbcac0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.643+0000 7fbcb4000640 1 -- 172.21.15.88:0/2105102827 shutdown_connections 2024-04-26T20:01:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.643+0000 7fbcb4000640 1 -- 172.21.15.88:0/2105102827 wait complete. 2024-04-26T20:01:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.644+0000 7fbcb4000640 1 Processor -- start 2024-04-26T20:01:10.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.644+0000 7fbcb4000640 1 -- start start 2024-04-26T20:01:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac0b4f50 0x7fbcac1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 0x7fbcac0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac15d1d0 con 0x7fbcac0cd650 2024-04-26T20:01:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbcac0b0240 con 0x7fbcac15cdf0 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb4000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbcac15d350 con 0x7fbcac0b4f50 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.645+0000 7fbcb0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58178/0 (socket says 172.21.15.88:58178) 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2776604180 learned_addr learned my addr 172.21.15.88:0/2776604180 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb1600640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac0b4f50 0x7fbcac1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac0b4f50 msgr2=0x7fbcac1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac0b4f50 0x7fbcac1480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 msgr2=0x7fbcac0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb2000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 0x7fbcac0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 0x7fbcac0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 -- 172.21.15.88:0/2776604180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcac0cee80 con 0x7fbcac0cd650 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb2000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 0x7fbcac0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.646+0000 7fbcb0c00640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc98000c00 tx=0x7fbc980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.647+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98041020 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.647+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc980306c0 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.647+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98037040 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.647+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcac0cf160 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.647+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcac14eb60 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.648+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcac14f050 con 0x7fbcac0cd650 2024-04-26T20:01:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.649+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc98044020 con 0x7fbcac0cd650 2024-04-26T20:01:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.650+0000 7fbca9600640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 0x7fbc8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.650+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbc980ac340 con 0x7fbcac0cd650 2024-04-26T20:01:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.650+0000 7fbcb1600640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 0x7fbc8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.651+0000 7fbcb1600640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 0x7fbc8c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbca0006fd0 tx=0x7fbca0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:10.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.655+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc9804b030 con 0x7fbcac0cd650 2024-04-26T20:01:10.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.835+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcac069540 con 0x7fbc8c06c8f0 2024-04-26T20:01:10.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.843+0000 7fbca9600640 1 -- 172.21.15.88:0/2776604180 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246631 (secure 0 0 0) 0x7fbcac069540 con 0x7fbc8c06c8f0 2024-04-26T20:01:10.845 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 msgr2=0x7fbc8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 0x7fbc8c06edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbca0006fd0 tx=0x7fbca0008040 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 msgr2=0x7fbcac0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc98000c00 tx=0x7fbc980046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 shutdown_connections 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcac15cdf0 0x7fbcac0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcac0cd650 0x7fbcac0cdfe0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc8c06c8f0 0x7fbc8c06edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 --2- 172.21.15.88:0/2776604180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbcac0b4f50 0x7fbcac1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 >> 172.21.15.88:0/2776604180 conn(0x7fbcac0b2d50 msgr2=0x7fbcac151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 shutdown_connections 2024-04-26T20:01:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:10.849+0000 7fbcb4000640 1 -- 172.21.15.88:0/2776604180 wait complete. 2024-04-26T20:01:10.868 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T20:01:08.965073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219956,"kb_used_data":3784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754764,"statfs":{"total":773094113280,"available":772868878336,"internally_reserved":0,"allocated":3874816,"data_stored":2200610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.106416"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886701+0000","last_change":"2024-04-26T20:01:04.886701+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886701+0000","last_undegraded":"2024-04-26T20:01:04.886701+0000","last_fullsized":"2024-04-26T20:01:04.886701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.896075+0000","last_change":"2024-04-26T20:01:04.896075+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.896075+0000","last_undegraded":"2024-04-26T20:01:04.896075+0000","last_fullsized":"2024-04-26T20:01:04.896075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880267+0000","last_change":"2024-04-26T20:01:04.880267+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880267+0000","last_undegraded":"2024-04-26T20:01:04.880267+0000","last_fullsized":"2024-04-26T20:01:04.880267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880742+0000","last_change":"2024-04-26T20:01:04.880742+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880742+0000","last_undegraded":"2024-04-26T20:01:04.880742+0000","last_fullsized":"2024-04-26T20:01:04.880742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872740+0000","last_change":"2024-04-26T20:01:04.872740+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872740+0000","last_undegraded":"2024-04-26T20:01:04.872740+0000","last_fullsized":"2024-04-26T20:01:04.872740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884638+0000","last_change":"2024-04-26T20:01:04.884638+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884638+0000","last_undegraded":"2024-04-26T20:01:04.884638+0000","last_fullsized":"2024-04-26T20:01:04.884638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889717+0000","last_change":"2024-04-26T20:01:04.889717+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889717+0000","last_undegraded":"2024-04-26T20:01:04.889717+0000","last_fullsized":"2024-04-26T20:01:04.889717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.895162+0000","last_change":"2024-04-26T20:01:04.895162+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.895162+0000","last_undegraded":"2024-04-26T20:01:04.895162+0000","last_fullsized":"2024-04-26T20:01:04.895162+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876488+0000","last_change":"2024-04-26T20:01:04.876488+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876488+0000","last_undegraded":"2024-04-26T20:01:04.876488+0000","last_fullsized":"2024-04-26T20:01:04.876488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878397+0000","last_change":"2024-04-26T20:01:04.878397+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878397+0000","last_undegraded":"2024-04-26T20:01:04.878397+0000","last_fullsized":"2024-04-26T20:01:04.878397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.892827+0000","last_change":"2024-04-26T20:01:04.892827+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.892827+0000","last_undegraded":"2024-04-26T20:01:04.892827+0000","last_fullsized":"2024-04-26T20:01:04.892827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.881971+0000","last_change":"2024-04-26T20:01:04.881971+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.881971+0000","last_undegraded":"2024-04-26T20:01:04.881971+0000","last_fullsized":"2024-04-26T20:01:04.881971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871892+0000","last_change":"2024-04-26T20:01:04.871892+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871892+0000","last_undegraded":"2024-04-26T20:01:04.871892+0000","last_fullsized":"2024-04-26T20:01:04.871892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884048+0000","last_change":"2024-04-26T20:01:04.884048+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884048+0000","last_undegraded":"2024-04-26T20:01:04.884048+0000","last_fullsized":"2024-04-26T20:01:04.884048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884458+0000","last_change":"2024-04-26T20:01:04.884458+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884458+0000","last_undegraded":"2024-04-26T20:01:04.884458+0000","last_fullsized":"2024-04-26T20:01:04.884458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.891186+0000","last_change":"2024-04-26T20:01:04.891186+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.891186+0000","last_undegraded":"2024-04-26T20:01:04.891186+0000","last_fullsized":"2024-04-26T20:01:04.891186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.893989+0000","last_change":"2024-04-26T20:01:04.893989+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.893989+0000","last_undegraded":"2024-04-26T20:01:04.893989+0000","last_fullsized":"2024-04-26T20:01:04.893989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889725+0000","last_change":"2024-04-26T20:01:04.889725+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889725+0000","last_undegraded":"2024-04-26T20:01:04.889725+0000","last_fullsized":"2024-04-26T20:01:04.889725+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888816+0000","last_change":"2024-04-26T20:01:04.888816+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888816+0000","last_undegraded":"2024-04-26T20:01:04.888816+0000","last_fullsized":"2024-04-26T20:01:04.888816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871091+0000","last_change":"2024-04-26T20:01:04.871091+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871091+0000","last_undegraded":"2024-04-26T20:01:04.871091+0000","last_fullsized":"2024-04-26T20:01:04.871091+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878594+0000","last_change":"2024-04-26T20:01:04.878594+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878594+0000","last_undegraded":"2024-04-26T20:01:04.878594+0000","last_fullsized":"2024-04-26T20:01:04.878594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.874045+0000","last_change":"2024-04-26T20:01:04.874045+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.874045+0000","last_undegraded":"2024-04-26T20:01:04.874045+0000","last_fullsized":"2024-04-26T20:01:04.874045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.879603+0000","last_change":"2024-04-26T20:01:04.879603+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.879603+0000","last_undegraded":"2024-04-26T20:01:04.879603+0000","last_fullsized":"2024-04-26T20:01:04.879603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880789+0000","last_change":"2024-04-26T20:01:04.880789+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880789+0000","last_undegraded":"2024-04-26T20:01:04.880789+0000","last_fullsized":"2024-04-26T20:01:04.880789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878875+0000","last_change":"2024-04-26T20:01:04.878875+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878875+0000","last_undegraded":"2024-04-26T20:01:04.878875+0000","last_fullsized":"2024-04-26T20:01:04.878875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886270+0000","last_change":"2024-04-26T20:01:04.886270+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886270+0000","last_undegraded":"2024-04-26T20:01:04.886270+0000","last_fullsized":"2024-04-26T20:01:04.886270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.878653+0000","last_change":"2024-04-26T20:01:04.878653+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.878653+0000","last_undegraded":"2024-04-26T20:01:04.878653+0000","last_fullsized":"2024-04-26T20:01:04.878653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880229+0000","last_change":"2024-04-26T20:01:04.880229+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880229+0000","last_undegraded":"2024-04-26T20:01:04.880229+0000","last_fullsized":"2024-04-26T20:01:04.880229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876370+0000","last_change":"2024-04-26T20:01:04.876370+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876370+0000","last_undegraded":"2024-04-26T20:01:04.876370+0000","last_fullsized":"2024-04-26T20:01:04.876370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889894+0000","last_change":"2024-04-26T20:01:04.889894+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889894+0000","last_undegraded":"2024-04-26T20:01:04.889894+0000","last_fullsized":"2024-04-26T20:01:04.889894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888045+0000","last_change":"2024-04-26T20:01:04.888045+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888045+0000","last_undegraded":"2024-04-26T20:01:04.888045+0000","last_fullsized":"2024-04-26T20:01:04.888045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.894702+0000","last_change":"2024-04-26T20:01:04.894702+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.894702+0000","last_undegraded":"2024-04-26T20:01:04.894702+0000","last_fullsized":"2024-04-26T20:01:04.894702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.879971+0000","last_change":"2024-04-26T20:01:04.879971+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.879971+0000","last_undegraded":"2024-04-26T20:01:04.879971+0000","last_fullsized":"2024-04-26T20:01:04.879971+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905028+0000","last_change":"2024-04-26T20:01:04.900074+0000","last_active":"2024-04-26T20:01:04.905028+0000","last_peered":"2024-04-26T20:01:04.905028+0000","last_clean":"2024-04-26T20:01:04.905028+0000","last_became_active":"2024-04-26T20:01:04.898252+0000","last_became_peered":"2024-04-26T20:01:04.898252+0000","last_unstale":"2024-04-26T20:01:04.905028+0000","last_undegraded":"2024-04-26T20:01:04.905028+0000","last_fullsized":"2024-04-26T20:01:04.905028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885982+0000","last_change":"2024-04-26T20:01:04.885982+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885982+0000","last_undegraded":"2024-04-26T20:01:04.885982+0000","last_fullsized":"2024-04-26T20:01:04.885982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904527+0000","last_change":"2024-04-26T20:01:04.902720+0000","last_active":"2024-04-26T20:01:04.904527+0000","last_peered":"2024-04-26T20:01:04.904527+0000","last_clean":"2024-04-26T20:01:04.904527+0000","last_became_active":"2024-04-26T20:01:04.902158+0000","last_became_peered":"2024-04-26T20:01:04.902158+0000","last_unstale":"2024-04-26T20:01:04.904527+0000","last_undegraded":"2024-04-26T20:01:04.904527+0000","last_fullsized":"2024-04-26T20:01:04.904527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.874062+0000","last_change":"2024-04-26T20:01:04.874062+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.874062+0000","last_undegraded":"2024-04-26T20:01:04.874062+0000","last_fullsized":"2024-04-26T20:01:04.874062+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.901408+0000","last_change":"2024-04-26T20:01:04.897236+0000","last_active":"2024-04-26T20:01:04.901408+0000","last_peered":"2024-04-26T20:01:04.901408+0000","last_clean":"2024-04-26T20:01:04.901408+0000","last_became_active":"2024-04-26T20:01:04.896622+0000","last_became_peered":"2024-04-26T20:01:04.896622+0000","last_unstale":"2024-04-26T20:01:04.901408+0000","last_undegraded":"2024-04-26T20:01:04.901408+0000","last_fullsized":"2024-04-26T20:01:04.901408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.877692+0000","last_change":"2024-04-26T20:01:04.877692+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.877692+0000","last_undegraded":"2024-04-26T20:01:04.877692+0000","last_fullsized":"2024-04-26T20:01:04.877692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.884619+0000","last_change":"2024-04-26T20:01:04.879892+0000","last_active":"2024-04-26T20:01:04.884619+0000","last_peered":"2024-04-26T20:01:04.884619+0000","last_clean":"2024-04-26T20:01:04.884619+0000","last_became_active":"2024-04-26T20:01:04.879459+0000","last_became_peered":"2024-04-26T20:01:04.879459+0000","last_unstale":"2024-04-26T20:01:04.884619+0000","last_undegraded":"2024-04-26T20:01:04.884619+0000","last_fullsized":"2024-04-26T20:01:04.884619+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:58:34.027761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889370+0000","last_change":"2024-04-26T20:01:04.889370+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889370+0000","last_undegraded":"2024-04-26T20:01:04.889370+0000","last_fullsized":"2024-04-26T20:01:04.889370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905145+0000","last_change":"2024-04-26T20:01:04.899358+0000","last_active":"2024-04-26T20:01:04.905145+0000","last_peered":"2024-04-26T20:01:04.905145+0000","last_clean":"2024-04-26T20:01:04.905145+0000","last_became_active":"2024-04-26T20:01:04.899066+0000","last_became_peered":"2024-04-26T20:01:04.899066+0000","last_unstale":"2024-04-26T20:01:04.905145+0000","last_undegraded":"2024-04-26T20:01:04.905145+0000","last_fullsized":"2024-04-26T20:01:04.905145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:32.524975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880387+0000","last_change":"2024-04-26T20:01:04.880387+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880387+0000","last_undegraded":"2024-04-26T20:01:04.880387+0000","last_fullsized":"2024-04-26T20:01:04.880387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.903001+0000","last_change":"2024-04-26T20:01:04.896700+0000","last_active":"2024-04-26T20:01:04.903001+0000","last_peered":"2024-04-26T20:01:04.903001+0000","last_clean":"2024-04-26T20:01:04.903001+0000","last_became_active":"2024-04-26T20:01:04.896384+0000","last_became_peered":"2024-04-26T20:01:04.896384+0000","last_unstale":"2024-04-26T20:01:04.903001+0000","last_undegraded":"2024-04-26T20:01:04.903001+0000","last_fullsized":"2024-04-26T20:01:04.903001+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:13.813164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872903+0000","last_change":"2024-04-26T20:01:04.872903+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872903+0000","last_undegraded":"2024-04-26T20:01:04.872903+0000","last_fullsized":"2024-04-26T20:01:04.872903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.894812+0000","last_change":"2024-04-26T20:01:04.891168+0000","last_active":"2024-04-26T20:01:04.894812+0000","last_peered":"2024-04-26T20:01:04.894812+0000","last_clean":"2024-04-26T20:01:04.894812+0000","last_became_active":"2024-04-26T20:01:04.890897+0000","last_became_peered":"2024-04-26T20:01:04.890897+0000","last_unstale":"2024-04-26T20:01:04.894812+0000","last_undegraded":"2024-04-26T20:01:04.894812+0000","last_fullsized":"2024-04-26T20:01:04.894812+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876486+0000","last_change":"2024-04-26T20:01:04.876486+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876486+0000","last_undegraded":"2024-04-26T20:01:04.876486+0000","last_fullsized":"2024-04-26T20:01:04.876486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904980+0000","last_change":"2024-04-26T20:01:04.900028+0000","last_active":"2024-04-26T20:01:04.904980+0000","last_peered":"2024-04-26T20:01:04.904980+0000","last_clean":"2024-04-26T20:01:04.904980+0000","last_became_active":"2024-04-26T20:01:04.899740+0000","last_became_peered":"2024-04-26T20:01:04.899740+0000","last_unstale":"2024-04-26T20:01:04.904980+0000","last_undegraded":"2024-04-26T20:01:04.904980+0000","last_fullsized":"2024-04-26T20:01:04.904980+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:19.885170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880346+0000","last_change":"2024-04-26T20:01:04.880346+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880346+0000","last_undegraded":"2024-04-26T20:01:04.880346+0000","last_fullsized":"2024-04-26T20:01:04.880346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905563+0000","last_change":"2024-04-26T20:01:04.904429+0000","last_active":"2024-04-26T20:01:04.905563+0000","last_peered":"2024-04-26T20:01:04.905563+0000","last_clean":"2024-04-26T20:01:04.905563+0000","last_became_active":"2024-04-26T20:01:04.904127+0000","last_became_peered":"2024-04-26T20:01:04.904127+0000","last_unstale":"2024-04-26T20:01:04.905563+0000","last_undegraded":"2024-04-26T20:01:04.905563+0000","last_fullsized":"2024-04-26T20:01:04.905563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885324+0000","last_change":"2024-04-26T20:01:04.885324+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885324+0000","last_undegraded":"2024-04-26T20:01:04.885324+0000","last_fullsized":"2024-04-26T20:01:04.885324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904502+0000","last_change":"2024-04-26T20:01:04.902692+0000","last_active":"2024-04-26T20:01:04.904502+0000","last_peered":"2024-04-26T20:01:04.904502+0000","last_clean":"2024-04-26T20:01:04.904502+0000","last_became_active":"2024-04-26T20:01:04.901817+0000","last_became_peered":"2024-04-26T20:01:04.901817+0000","last_unstale":"2024-04-26T20:01:04.904502+0000","last_undegraded":"2024-04-26T20:01:04.904502+0000","last_fullsized":"2024-04-26T20:01:04.904502+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889418+0000","last_change":"2024-04-26T20:01:04.889418+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889418+0000","last_undegraded":"2024-04-26T20:01:04.889418+0000","last_fullsized":"2024-04-26T20:01:04.889418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905160+0000","last_change":"2024-04-26T20:01:04.903315+0000","last_active":"2024-04-26T20:01:04.905160+0000","last_peered":"2024-04-26T20:01:04.905160+0000","last_clean":"2024-04-26T20:01:04.905160+0000","last_became_active":"2024-04-26T20:01:04.902750+0000","last_became_peered":"2024-04-26T20:01:04.902750+0000","last_unstale":"2024-04-26T20:01:04.905160+0000","last_undegraded":"2024-04-26T20:01:04.905160+0000","last_fullsized":"2024-04-26T20:01:04.905160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.883743+0000","last_change":"2024-04-26T20:01:04.883743+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.883743+0000","last_undegraded":"2024-04-26T20:01:04.883743+0000","last_fullsized":"2024-04-26T20:01:04.883743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900539+0000","last_change":"2024-04-26T20:01:04.896296+0000","last_active":"2024-04-26T20:01:04.900539+0000","last_peered":"2024-04-26T20:01:04.900539+0000","last_clean":"2024-04-26T20:01:04.900539+0000","last_became_active":"2024-04-26T20:01:04.896034+0000","last_became_peered":"2024-04-26T20:01:04.896034+0000","last_unstale":"2024-04-26T20:01:04.900539+0000","last_undegraded":"2024-04-26T20:01:04.900539+0000","last_fullsized":"2024-04-26T20:01:04.900539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876389+0000","last_change":"2024-04-26T20:01:04.876389+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876389+0000","last_undegraded":"2024-04-26T20:01:04.876389+0000","last_fullsized":"2024-04-26T20:01:04.876389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905952+0000","last_change":"2024-04-26T20:01:04.902304+0000","last_active":"2024-04-26T20:01:04.905952+0000","last_peered":"2024-04-26T20:01:04.905952+0000","last_clean":"2024-04-26T20:01:04.905952+0000","last_became_active":"2024-04-26T20:01:04.901986+0000","last_became_peered":"2024-04-26T20:01:04.901986+0000","last_unstale":"2024-04-26T20:01:04.905952+0000","last_undegraded":"2024-04-26T20:01:04.905952+0000","last_fullsized":"2024-04-26T20:01:04.905952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:33.368463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.890900+0000","last_change":"2024-04-26T20:01:04.890900+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.890900+0000","last_undegraded":"2024-04-26T20:01:04.890900+0000","last_fullsized":"2024-04-26T20:01:04.890900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900398+0000","last_change":"2024-04-26T20:01:04.898231+0000","last_active":"2024-04-26T20:01:04.900398+0000","last_peered":"2024-04-26T20:01:04.900398+0000","last_clean":"2024-04-26T20:01:04.900398+0000","last_became_active":"2024-04-26T20:01:04.896868+0000","last_became_peered":"2024-04-26T20:01:04.896868+0000","last_unstale":"2024-04-26T20:01:04.900398+0000","last_undegraded":"2024-04-26T20:01:04.900398+0000","last_fullsized":"2024-04-26T20:01:04.900398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.892352+0000","last_change":"2024-04-26T20:01:04.892352+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.892352+0000","last_undegraded":"2024-04-26T20:01:04.892352+0000","last_fullsized":"2024-04-26T20:01:04.892352+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905254+0000","last_change":"2024-04-26T20:01:04.904256+0000","last_active":"2024-04-26T20:01:04.905254+0000","last_peered":"2024-04-26T20:01:04.905254+0000","last_clean":"2024-04-26T20:01:04.905254+0000","last_became_active":"2024-04-26T20:01:04.903998+0000","last_became_peered":"2024-04-26T20:01:04.903998+0000","last_unstale":"2024-04-26T20:01:04.905254+0000","last_undegraded":"2024-04-26T20:01:04.905254+0000","last_fullsized":"2024-04-26T20:01:04.905254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.886188+0000","last_change":"2024-04-26T20:01:04.886188+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.886188+0000","last_undegraded":"2024-04-26T20:01:04.886188+0000","last_fullsized":"2024-04-26T20:01:04.886188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897458+0000","last_change":"2024-04-26T20:01:04.894197+0000","last_active":"2024-04-26T20:01:04.897458+0000","last_peered":"2024-04-26T20:01:04.897458+0000","last_clean":"2024-04-26T20:01:04.897458+0000","last_became_active":"2024-04-26T20:01:04.893919+0000","last_became_peered":"2024-04-26T20:01:04.893919+0000","last_unstale":"2024-04-26T20:01:04.897458+0000","last_undegraded":"2024-04-26T20:01:04.897458+0000","last_fullsized":"2024-04-26T20:01:04.897458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876928+0000","last_change":"2024-04-26T20:01:04.876928+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876928+0000","last_undegraded":"2024-04-26T20:01:04.876928+0000","last_fullsized":"2024-04-26T20:01:04.876928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900644+0000","last_change":"2024-04-26T20:01:04.898895+0000","last_active":"2024-04-26T20:01:04.900644+0000","last_peered":"2024-04-26T20:01:04.900644+0000","last_clean":"2024-04-26T20:01:04.900644+0000","last_became_active":"2024-04-26T20:01:04.898535+0000","last_became_peered":"2024-04-26T20:01:04.898535+0000","last_unstale":"2024-04-26T20:01:04.900644+0000","last_undegraded":"2024-04-26T20:01:04.900644+0000","last_fullsized":"2024-04-26T20:01:04.900644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:09.553490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.885655+0000","last_change":"2024-04-26T20:01:04.885655+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.885655+0000","last_undegraded":"2024-04-26T20:01:04.885655+0000","last_fullsized":"2024-04-26T20:01:04.885655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905993+0000","last_change":"2024-04-26T20:01:04.902861+0000","last_active":"2024-04-26T20:01:04.905993+0000","last_peered":"2024-04-26T20:01:04.905993+0000","last_clean":"2024-04-26T20:01:04.905993+0000","last_became_active":"2024-04-26T20:01:04.902393+0000","last_became_peered":"2024-04-26T20:01:04.902393+0000","last_unstale":"2024-04-26T20:01:04.905993+0000","last_undegraded":"2024-04-26T20:01:04.905993+0000","last_fullsized":"2024-04-26T20:01:04.905993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:05.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.876312+0000","last_change":"2024-04-26T20:01:04.876312+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.876312+0000","last_undegraded":"2024-04-26T20:01:04.876312+0000","last_fullsized":"2024-04-26T20:01:04.876312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897922+0000","last_change":"2024-04-26T20:01:04.887577+0000","last_active":"2024-04-26T20:01:04.897922+0000","last_peered":"2024-04-26T20:01:04.897922+0000","last_clean":"2024-04-26T20:01:04.897922+0000","last_became_active":"2024-04-26T20:01:04.887059+0000","last_became_peered":"2024-04-26T20:01:04.887059+0000","last_unstale":"2024-04-26T20:01:04.897922+0000","last_undegraded":"2024-04-26T20:01:04.897922+0000","last_fullsized":"2024-04-26T20:01:04.897922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:13.994097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872448+0000","last_change":"2024-04-26T20:01:04.872448+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872448+0000","last_undegraded":"2024-04-26T20:01:04.872448+0000","last_fullsized":"2024-04-26T20:01:04.872448+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.904682+0000","last_change":"2024-04-26T20:01:04.897364+0000","last_active":"2024-04-26T20:01:04.904682+0000","last_peered":"2024-04-26T20:01:04.904682+0000","last_clean":"2024-04-26T20:01:04.904682+0000","last_became_active":"2024-04-26T20:01:04.896186+0000","last_became_peered":"2024-04-26T20:01:04.896186+0000","last_unstale":"2024-04-26T20:01:04.904682+0000","last_undegraded":"2024-04-26T20:01:04.904682+0000","last_fullsized":"2024-04-26T20:01:04.904682+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:58:34.027761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.870312+0000","last_change":"2024-04-26T20:01:04.870312+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.870312+0000","last_undegraded":"2024-04-26T20:01:04.870312+0000","last_fullsized":"2024-04-26T20:01:04.870312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.891773+0000","last_change":"2024-04-26T20:01:04.889881+0000","last_active":"2024-04-26T20:01:04.891773+0000","last_peered":"2024-04-26T20:01:04.891773+0000","last_clean":"2024-04-26T20:01:04.891773+0000","last_became_active":"2024-04-26T20:01:04.889623+0000","last_became_peered":"2024-04-26T20:01:04.889623+0000","last_unstale":"2024-04-26T20:01:04.891773+0000","last_undegraded":"2024-04-26T20:01:04.891773+0000","last_fullsized":"2024-04-26T20:01:04.891773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.871118+0000","last_change":"2024-04-26T20:01:04.871118+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.871118+0000","last_undegraded":"2024-04-26T20:01:04.871118+0000","last_fullsized":"2024-04-26T20:01:04.871118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905070+0000","last_change":"2024-04-26T20:01:04.901055+0000","last_active":"2024-04-26T20:01:04.905070+0000","last_peered":"2024-04-26T20:01:04.905070+0000","last_clean":"2024-04-26T20:01:04.905070+0000","last_became_active":"2024-04-26T20:01:04.900083+0000","last_became_peered":"2024-04-26T20:01:04.900083+0000","last_unstale":"2024-04-26T20:01:04.905070+0000","last_undegraded":"2024-04-26T20:01:04.905070+0000","last_fullsized":"2024-04-26T20:01:04.905070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:19.030356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.872842+0000","last_change":"2024-04-26T20:01:04.872842+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.872842+0000","last_undegraded":"2024-04-26T20:01:04.872842+0000","last_fullsized":"2024-04-26T20:01:04.872842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.897983+0000","last_change":"2024-04-26T20:01:04.888976+0000","last_active":"2024-04-26T20:01:04.897983+0000","last_peered":"2024-04-26T20:01:04.897983+0000","last_clean":"2024-04-26T20:01:04.897983+0000","last_became_active":"2024-04-26T20:01:04.888367+0000","last_became_peered":"2024-04-26T20:01:04.888367+0000","last_unstale":"2024-04-26T20:01:04.897983+0000","last_undegraded":"2024-04-26T20:01:04.897983+0000","last_fullsized":"2024-04-26T20:01:04.897983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:37.864035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.890176+0000","last_change":"2024-04-26T20:01:04.890176+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.890176+0000","last_undegraded":"2024-04-26T20:01:04.890176+0000","last_fullsized":"2024-04-26T20:01:04.890176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.910725+0000","last_change":"2024-04-26T20:01:04.906119+0000","last_active":"2024-04-26T20:01:04.910725+0000","last_peered":"2024-04-26T20:01:04.910725+0000","last_clean":"2024-04-26T20:01:04.910725+0000","last_became_active":"2024-04-26T20:01:04.905793+0000","last_became_peered":"2024-04-26T20:01:04.905793+0000","last_unstale":"2024-04-26T20:01:04.910725+0000","last_undegraded":"2024-04-26T20:01:04.910725+0000","last_fullsized":"2024-04-26T20:01:04.910725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.870789+0000","last_change":"2024-04-26T20:01:04.870789+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.870789+0000","last_undegraded":"2024-04-26T20:01:04.870789+0000","last_fullsized":"2024-04-26T20:01:04.870789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900429+0000","last_change":"2024-04-26T20:01:04.897905+0000","last_active":"2024-04-26T20:01:04.900429+0000","last_peered":"2024-04-26T20:01:04.900429+0000","last_clean":"2024-04-26T20:01:04.900429+0000","last_became_active":"2024-04-26T20:01:04.897556+0000","last_became_peered":"2024-04-26T20:01:04.897556+0000","last_unstale":"2024-04-26T20:01:04.900429+0000","last_undegraded":"2024-04-26T20:01:04.900429+0000","last_fullsized":"2024-04-26T20:01:04.900429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.884382+0000","last_change":"2024-04-26T20:01:04.884382+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.884382+0000","last_undegraded":"2024-04-26T20:01:04.884382+0000","last_fullsized":"2024-04-26T20:01:04.884382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.890435+0000","last_change":"2024-04-26T20:01:04.888031+0000","last_active":"2024-04-26T20:01:04.890435+0000","last_peered":"2024-04-26T20:01:04.890435+0000","last_clean":"2024-04-26T20:01:04.890435+0000","last_became_active":"2024-04-26T20:01:04.887522+0000","last_became_peered":"2024-04-26T20:01:04.887522+0000","last_unstale":"2024-04-26T20:01:04.890435+0000","last_undegraded":"2024-04-26T20:01:04.890435+0000","last_fullsized":"2024-04-26T20:01:04.890435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880299+0000","last_change":"2024-04-26T20:01:04.880299+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880299+0000","last_undegraded":"2024-04-26T20:01:04.880299+0000","last_fullsized":"2024-04-26T20:01:04.880299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.887119+0000","last_change":"2024-04-26T20:01:04.882527+0000","last_active":"2024-04-26T20:01:04.887119+0000","last_peered":"2024-04-26T20:01:04.887119+0000","last_clean":"2024-04-26T20:01:04.887119+0000","last_became_active":"2024-04-26T20:01:04.881905+0000","last_became_peered":"2024-04-26T20:01:04.881905+0000","last_unstale":"2024-04-26T20:01:04.887119+0000","last_undegraded":"2024-04-26T20:01:04.887119+0000","last_fullsized":"2024-04-26T20:01:04.887119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.888846+0000","last_change":"2024-04-26T20:01:04.888846+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.888846+0000","last_undegraded":"2024-04-26T20:01:04.888846+0000","last_fullsized":"2024-04-26T20:01:04.888846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900718+0000","last_change":"2024-04-26T20:01:04.898199+0000","last_active":"2024-04-26T20:01:04.900718+0000","last_peered":"2024-04-26T20:01:04.900718+0000","last_clean":"2024-04-26T20:01:04.900718+0000","last_became_active":"2024-04-26T20:01:04.897910+0000","last_became_peered":"2024-04-26T20:01:04.897910+0000","last_unstale":"2024-04-26T20:01:04.900718+0000","last_undegraded":"2024-04-26T20:01:04.900718+0000","last_fullsized":"2024-04-26T20:01:04.900718+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880280+0000","last_change":"2024-04-26T20:01:04.880280+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880280+0000","last_undegraded":"2024-04-26T20:01:04.880280+0000","last_fullsized":"2024-04-26T20:01:04.880280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.900425+0000","last_change":"2024-04-26T20:01:04.898465+0000","last_active":"2024-04-26T20:01:04.900425+0000","last_peered":"2024-04-26T20:01:04.900425+0000","last_clean":"2024-04-26T20:01:04.900425+0000","last_became_active":"2024-04-26T20:01:04.897963+0000","last_became_peered":"2024-04-26T20:01:04.897963+0000","last_unstale":"2024-04-26T20:01:04.900425+0000","last_undegraded":"2024-04-26T20:01:04.900425+0000","last_fullsized":"2024-04-26T20:01:04.900425+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.880393+0000","last_change":"2024-04-26T20:01:04.880393+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.880393+0000","last_undegraded":"2024-04-26T20:01:04.880393+0000","last_fullsized":"2024-04-26T20:01:04.880393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.901497+0000","last_change":"2024-04-26T20:01:04.894822+0000","last_active":"2024-04-26T20:01:04.901497+0000","last_peered":"2024-04-26T20:01:04.901497+0000","last_clean":"2024-04-26T20:01:04.901497+0000","last_became_active":"2024-04-26T20:01:04.893421+0000","last_became_peered":"2024-04-26T20:01:04.893421+0000","last_unstale":"2024-04-26T20:01:04.901497+0000","last_undegraded":"2024-04-26T20:01:04.901497+0000","last_fullsized":"2024-04-26T20:01:04.901497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:42.345562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.877406+0000","last_change":"2024-04-26T20:01:04.877406+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.877406+0000","last_undegraded":"2024-04-26T20:01:04.877406+0000","last_fullsized":"2024-04-26T20:01:04.877406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905190+0000","last_change":"2024-04-26T20:01:04.904249+0000","last_active":"2024-04-26T20:01:04.905190+0000","last_peered":"2024-04-26T20:01:04.905190+0000","last_clean":"2024-04-26T20:01:04.905190+0000","last_became_active":"2024-04-26T20:01:04.903995+0000","last_became_peered":"2024-04-26T20:01:04.903995+0000","last_unstale":"2024-04-26T20:01:04.905190+0000","last_undegraded":"2024-04-26T20:01:04.905190+0000","last_fullsized":"2024-04-26T20:01:04.905190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:01:04.889392+0000","last_change":"2024-04-26T20:01:04.889392+0000","last_active":"2024-04-26T20:01:04.857041+0000","last_peered":"2024-04-26T20:01:04.857041+0000","last_clean":"2024-04-26T20:01:04.857041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:01:04.889392+0000","last_undegraded":"2024-04-26T20:01:04.889392+0000","last_fullsized":"2024-04-26T20:01:04.889392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.905888+0000","last_change":"2024-04-26T20:01:04.903742+0000","last_active":"2024-04-26T20:01:04.905888+0000","last_peered":"2024-04-26T20:01:04.905888+0000","last_clean":"2024-04-26T20:01:04.905888+0000","last_became_active":"2024-04-26T20:01:04.903299+0000","last_became_peered":"2024-04-26T20:01:04.903299+0000","last_unstale":"2024-04-26T20:01:04.905888+0000","last_undegraded":"2024-04-26T20:01:04.905888+0000","last_fullsized":"2024-04-26T20:01:04.905888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:07.281520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T20:01:04.899769+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:04.899769+0000","last_peered":"2024-04-26T20:01:04.899769+0000","last_clean":"2024-04-26T20:01:04.899769+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:04.899769+0000","last_undegraded":"2024-04-26T20:01:04.899769+0000","last_fullsized":"2024-04-26T20:01:04.899769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":711979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:01:10.872 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:01:11.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.104+0000 7f05cda00640 1 Processor -- start 2024-04-26T20:01:11.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.104+0000 7f05cda00640 1 -- start start 2024-04-26T20:01:11.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cda00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 0x7f05c80b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cda00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c8146a60 con 0x7f05c80b4b30 2024-04-26T20:01:11.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cda00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c8161750 con 0x7f05c80d0e70 2024-04-26T20:01:11.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cda00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f05c806e970 con 0x7f05c8146680 2024-04-26T20:01:11.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cd000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f05c80b4b30 0x7f05c8161040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34344/0 (socket says 172.21.15.88:34344) 2024-04-26T20:01:11.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.106+0000 7f05cd000640 1 -- 172.21.15.88:0/2742096838 learned_addr learned my addr 172.21.15.88:0/2742096838 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:11.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7e00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 0x7f05c80b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455160063 0 0) 0x7f05c8146a60 con 0x7f05c80b4b30 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05ac003520 con 0x7f05c80b4b30 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1997769101 0 0) 0x7f05ac003520 con 0x7f05c80b4b30 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 msgr2=0x7f05c80b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 0x7f05c80b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 >> v1:172.21.15.132:6789/0 conn(0x7f05c80d0e70 legacy=0x7f05c8160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c806dd60 con 0x7f05c80b4b30 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f05c00019f0 con 0x7f05c80b4b30 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 >> v1:172.21.15.88:6789/0 conn(0x7f05c80b4b30 legacy=0x7f05c8161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0039e0 0x7f05ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 0x7f05ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac004d40 0x7f05ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac00ba70 con 0x7f05ac0039e0 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac00bbf0 con 0x7f05ac004d40 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac00bdf0 con 0x7f05c8146680 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.107+0000 7f05c7e00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0039e0 0x7f05ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.108+0000 7f05c7e00640 1 -- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 msgr2=0x7f05ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.108+0000 7f05c7e00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c8146680 0x7f05ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.108+0000 7f05c7e00640 1 -- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac004d40 msgr2=0x7f05ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.108+0000 7f05c7e00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac004d40 0x7f05ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.108+0000 7f05c7e00640 1 -- 172.21.15.88:0/2742096838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac003520 con 0x7f05ac0039e0 2024-04-26T20:01:11.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.109+0000 7f05c7e00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0039e0 0x7f05ac003e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009a90 tx=0x7f05ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb3380b217f991c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.109+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8036070 con 0x7f05ac0039e0 2024-04-26T20:01:11.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.109+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b8031400 con 0x7f05ac0039e0 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.109+0000 7f05c7400640 1 -- 172.21.15.88:0/2742096838 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8009690 con 0x7f05ac0039e0 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 -- 172.21.15.88:0/2742096838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0039e0 msgr2=0x7f05ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 --2- 172.21.15.88:0/2742096838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0039e0 0x7f05ac003e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009a90 tx=0x7f05ac005830 comp rx=0 tx=0).stop 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cd000640 1 -- 172.21.15.88:0/2742096838 reap_dead start 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 -- 172.21.15.88:0/2742096838 shutdown_connections 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 -- 172.21.15.88:0/2742096838 >> 172.21.15.88:0/2742096838 conn(0x7f05c806c740 msgr2=0x7f05c806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 -- 172.21.15.88:0/2742096838 shutdown_connections 2024-04-26T20:01:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.110+0000 7f05cda00640 1 -- 172.21.15.88:0/2742096838 wait complete. 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.111+0000 7f05cda00640 1 Processor -- start 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 -- start start 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0039e0 0x7f05c813b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c81372b0 0x7f05c8137730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05c806e600 con 0x7f05c8144360 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f05c806e320 con 0x7f05ac0039e0 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.112+0000 7f05cda00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f05c80cd6a0 con 0x7f05c81372b0 2024-04-26T20:01:11.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58192/0 (socket says 172.21.15.88:58192) 2024-04-26T20:01:11.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 -- 172.21.15.88:0/1072888876 learned_addr learned my addr 172.21.15.88:0/1072888876 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05c7e00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0039e0 0x7f05c813b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 -- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c81372b0 msgr2=0x7f05c8137730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c81372b0 0x7f05c8137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 -- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0039e0 msgr2=0x7f05c813b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0039e0 0x7f05c813b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 -- 172.21.15.88:0/1072888876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c8137c70 con 0x7f05c8144360 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.113+0000 7f05cd000640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f05c80d1a10 tx=0x7f05c000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.114+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000cea0 con 0x7f05c8144360 2024-04-26T20:01:11.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.114+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c000a610 con 0x7f05c8144360 2024-04-26T20:01:11.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.114+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000aa00 con 0x7f05c8144360 2024-04-26T20:01:11.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.114+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81450e0 con 0x7f05c8144360 2024-04-26T20:01:11.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.114+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81390e0 con 0x7f05c8144360 2024-04-26T20:01:11.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.115+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c80693c0 con 0x7f05c8144360 2024-04-26T20:01:11.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.116+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05c000aba0 con 0x7f05c8144360 2024-04-26T20:01:11.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.117+0000 7f05c4c00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 0x7f05a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.117+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f05c008b490 con 0x7f05c8144360 2024-04-26T20:01:11.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.117+0000 7f05c7e00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 0x7f05a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.118+0000 7f05c7e00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 0x7f05a406ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05b8033000 tx=0x7f05b8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.121+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05c001a020 con 0x7f05c8144360 2024-04-26T20:01:11.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.301+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05c8140970 con 0x7f05a406c920 2024-04-26T20:01:11.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.309+0000 7f05c4c00640 1 -- 172.21.15.88:0/1072888876 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f05c8140970 con 0x7f05a406c920 2024-04-26T20:01:11.314 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 msgr2=0x7f05a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 0x7f05a406ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05b8033000 tx=0x7f05b8008030 comp rx=0 tx=0).stop 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 msgr2=0x7f05c81447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f05c80d1a10 tx=0x7f05c000cc50 comp rx=0 tx=0).stop 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 shutdown_connections 2024-04-26T20:01:11.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c8144360 0x7f05c81447e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f05a406c920 0x7f05a406ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f05c81372b0 0x7f05c8137730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 --2- 172.21.15.88:0/1072888876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0039e0 0x7f05c813b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 >> 172.21.15.88:0/1072888876 conn(0x7f05c806c740 msgr2=0x7f05c80b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 shutdown_connections 2024-04-26T20:01:11.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.315+0000 7f05cda00640 1 -- 172.21.15.88:0/1072888876 wait complete. 2024-04-26T20:01:11.334 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-26T20:01:10.965366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220340,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754380,"statfs":{"total":773094113280,"available":772868485120,"internally_reserved":0,"allocated":4235264,"data_stored":2359716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.099838"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422513+0000","last_change":"2024-04-26T20:01:05.919081+0000","last_active":"2024-04-26T20:01:07.422513+0000","last_peered":"2024-04-26T20:01:07.422513+0000","last_clean":"2024-04-26T20:01:07.422513+0000","last_became_active":"2024-04-26T20:01:05.916236+0000","last_became_peered":"2024-04-26T20:01:05.916236+0000","last_unstale":"2024-04-26T20:01:07.422513+0000","last_undegraded":"2024-04-26T20:01:07.422513+0000","last_fullsized":"2024-04-26T20:01:07.422513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.876764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421185+0000","last_change":"2024-04-26T20:01:05.924948+0000","last_active":"2024-04-26T20:01:07.421185+0000","last_peered":"2024-04-26T20:01:07.421185+0000","last_clean":"2024-04-26T20:01:07.421185+0000","last_became_active":"2024-04-26T20:01:05.924322+0000","last_became_peered":"2024-04-26T20:01:05.924322+0000","last_unstale":"2024-04-26T20:01:07.421185+0000","last_undegraded":"2024-04-26T20:01:07.421185+0000","last_fullsized":"2024-04-26T20:01:07.421185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:39.001253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422789+0000","last_change":"2024-04-26T20:01:05.925708+0000","last_active":"2024-04-26T20:01:07.422789+0000","last_peered":"2024-04-26T20:01:07.422789+0000","last_clean":"2024-04-26T20:01:07.422789+0000","last_became_active":"2024-04-26T20:01:05.924331+0000","last_became_peered":"2024-04-26T20:01:05.924331+0000","last_unstale":"2024-04-26T20:01:07.422789+0000","last_undegraded":"2024-04-26T20:01:07.422789+0000","last_fullsized":"2024-04-26T20:01:07.422789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930927+0000","last_change":"2024-04-26T20:01:05.930430+0000","last_active":"2024-04-26T20:01:06.930927+0000","last_peered":"2024-04-26T20:01:06.930927+0000","last_clean":"2024-04-26T20:01:06.930927+0000","last_became_active":"2024-04-26T20:01:05.930139+0000","last_became_peered":"2024-04-26T20:01:05.930139+0000","last_unstale":"2024-04-26T20:01:06.930927+0000","last_undegraded":"2024-04-26T20:01:06.930927+0000","last_fullsized":"2024-04-26T20:01:06.930927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928615+0000","last_change":"2024-04-26T20:01:05.934500+0000","last_active":"2024-04-26T20:01:06.928615+0000","last_peered":"2024-04-26T20:01:06.928615+0000","last_clean":"2024-04-26T20:01:06.928615+0000","last_became_active":"2024-04-26T20:01:05.933310+0000","last_became_peered":"2024-04-26T20:01:05.933310+0000","last_unstale":"2024-04-26T20:01:06.928615+0000","last_undegraded":"2024-04-26T20:01:06.928615+0000","last_fullsized":"2024-04-26T20:01:06.928615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420434+0000","last_change":"2024-04-26T20:01:05.912926+0000","last_active":"2024-04-26T20:01:07.420434+0000","last_peered":"2024-04-26T20:01:07.420434+0000","last_clean":"2024-04-26T20:01:07.420434+0000","last_became_active":"2024-04-26T20:01:05.912473+0000","last_became_peered":"2024-04-26T20:01:05.912473+0000","last_unstale":"2024-04-26T20:01:07.420434+0000","last_undegraded":"2024-04-26T20:01:07.420434+0000","last_fullsized":"2024-04-26T20:01:07.420434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:51.470568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423817+0000","last_change":"2024-04-26T20:01:05.928943+0000","last_active":"2024-04-26T20:01:07.423817+0000","last_peered":"2024-04-26T20:01:07.423817+0000","last_clean":"2024-04-26T20:01:07.423817+0000","last_became_active":"2024-04-26T20:01:05.928434+0000","last_became_peered":"2024-04-26T20:01:05.928434+0000","last_unstale":"2024-04-26T20:01:07.423817+0000","last_undegraded":"2024-04-26T20:01:07.423817+0000","last_fullsized":"2024-04-26T20:01:07.423817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421731+0000","last_change":"2024-04-26T20:01:05.940280+0000","last_active":"2024-04-26T20:01:07.421731+0000","last_peered":"2024-04-26T20:01:07.421731+0000","last_clean":"2024-04-26T20:01:07.421731+0000","last_became_active":"2024-04-26T20:01:05.940034+0000","last_became_peered":"2024-04-26T20:01:05.940034+0000","last_unstale":"2024-04-26T20:01:07.421731+0000","last_undegraded":"2024-04-26T20:01:07.421731+0000","last_fullsized":"2024-04-26T20:01:07.421731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929681+0000","last_change":"2024-04-26T20:01:05.945356+0000","last_active":"2024-04-26T20:01:06.929681+0000","last_peered":"2024-04-26T20:01:06.929681+0000","last_clean":"2024-04-26T20:01:06.929681+0000","last_became_active":"2024-04-26T20:01:05.945132+0000","last_became_peered":"2024-04-26T20:01:05.945132+0000","last_unstale":"2024-04-26T20:01:06.929681+0000","last_undegraded":"2024-04-26T20:01:06.929681+0000","last_fullsized":"2024-04-26T20:01:06.929681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931906+0000","last_change":"2024-04-26T20:01:05.926545+0000","last_active":"2024-04-26T20:01:06.931906+0000","last_peered":"2024-04-26T20:01:06.931906+0000","last_clean":"2024-04-26T20:01:06.931906+0000","last_became_active":"2024-04-26T20:01:05.924721+0000","last_became_peered":"2024-04-26T20:01:05.924721+0000","last_unstale":"2024-04-26T20:01:06.931906+0000","last_undegraded":"2024-04-26T20:01:06.931906+0000","last_fullsized":"2024-04-26T20:01:06.931906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424805+0000","last_change":"2024-04-26T20:01:05.939937+0000","last_active":"2024-04-26T20:01:07.424805+0000","last_peered":"2024-04-26T20:01:07.424805+0000","last_clean":"2024-04-26T20:01:07.424805+0000","last_became_active":"2024-04-26T20:01:05.939644+0000","last_became_peered":"2024-04-26T20:01:05.939644+0000","last_unstale":"2024-04-26T20:01:07.424805+0000","last_undegraded":"2024-04-26T20:01:07.424805+0000","last_fullsized":"2024-04-26T20:01:07.424805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423650+0000","last_change":"2024-04-26T20:01:05.919891+0000","last_active":"2024-04-26T20:01:07.423650+0000","last_peered":"2024-04-26T20:01:07.423650+0000","last_clean":"2024-04-26T20:01:07.423650+0000","last_became_active":"2024-04-26T20:01:05.919631+0000","last_became_peered":"2024-04-26T20:01:05.919631+0000","last_unstale":"2024-04-26T20:01:07.423650+0000","last_undegraded":"2024-04-26T20:01:07.423650+0000","last_fullsized":"2024-04-26T20:01:07.423650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928767+0000","last_change":"2024-04-26T20:01:05.927158+0000","last_active":"2024-04-26T20:01:06.928767+0000","last_peered":"2024-04-26T20:01:06.928767+0000","last_clean":"2024-04-26T20:01:06.928767+0000","last_became_active":"2024-04-26T20:01:05.926783+0000","last_became_peered":"2024-04-26T20:01:05.926783+0000","last_unstale":"2024-04-26T20:01:06.928767+0000","last_undegraded":"2024-04-26T20:01:06.928767+0000","last_fullsized":"2024-04-26T20:01:06.928767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:01.310367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420598+0000","last_change":"2024-04-26T20:01:05.940579+0000","last_active":"2024-04-26T20:01:07.420598+0000","last_peered":"2024-04-26T20:01:07.420598+0000","last_clean":"2024-04-26T20:01:07.420598+0000","last_became_active":"2024-04-26T20:01:05.940304+0000","last_became_peered":"2024-04-26T20:01:05.940304+0000","last_unstale":"2024-04-26T20:01:07.420598+0000","last_undegraded":"2024-04-26T20:01:07.420598+0000","last_fullsized":"2024-04-26T20:01:07.420598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:21.271559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421597+0000","last_change":"2024-04-26T20:01:05.923735+0000","last_active":"2024-04-26T20:01:07.421597+0000","last_peered":"2024-04-26T20:01:07.421597+0000","last_clean":"2024-04-26T20:01:07.421597+0000","last_became_active":"2024-04-26T20:01:05.923381+0000","last_became_peered":"2024-04-26T20:01:05.923381+0000","last_unstale":"2024-04-26T20:01:07.421597+0000","last_undegraded":"2024-04-26T20:01:07.421597+0000","last_fullsized":"2024-04-26T20:01:07.421597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423136+0000","last_change":"2024-04-26T20:01:05.923995+0000","last_active":"2024-04-26T20:01:07.423136+0000","last_peered":"2024-04-26T20:01:07.423136+0000","last_clean":"2024-04-26T20:01:07.423136+0000","last_became_active":"2024-04-26T20:01:05.923638+0000","last_became_peered":"2024-04-26T20:01:05.923638+0000","last_unstale":"2024-04-26T20:01:07.423136+0000","last_undegraded":"2024-04-26T20:01:07.423136+0000","last_fullsized":"2024-04-26T20:01:07.423136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424734+0000","last_change":"2024-04-26T20:01:05.927247+0000","last_active":"2024-04-26T20:01:07.424734+0000","last_peered":"2024-04-26T20:01:07.424734+0000","last_clean":"2024-04-26T20:01:07.424734+0000","last_became_active":"2024-04-26T20:01:05.924925+0000","last_became_peered":"2024-04-26T20:01:05.924925+0000","last_unstale":"2024-04-26T20:01:07.424734+0000","last_undegraded":"2024-04-26T20:01:07.424734+0000","last_fullsized":"2024-04-26T20:01:07.424734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421254+0000","last_change":"2024-04-26T20:01:05.926554+0000","last_active":"2024-04-26T20:01:07.421254+0000","last_peered":"2024-04-26T20:01:07.421254+0000","last_clean":"2024-04-26T20:01:07.421254+0000","last_became_active":"2024-04-26T20:01:05.926287+0000","last_became_peered":"2024-04-26T20:01:05.926287+0000","last_unstale":"2024-04-26T20:01:07.421254+0000","last_undegraded":"2024-04-26T20:01:07.421254+0000","last_fullsized":"2024-04-26T20:01:07.421254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:43.379844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421059+0000","last_change":"2024-04-26T20:01:05.914996+0000","last_active":"2024-04-26T20:01:07.421059+0000","last_peered":"2024-04-26T20:01:07.421059+0000","last_clean":"2024-04-26T20:01:07.421059+0000","last_became_active":"2024-04-26T20:01:05.914692+0000","last_became_peered":"2024-04-26T20:01:05.914692+0000","last_unstale":"2024-04-26T20:01:07.421059+0000","last_undegraded":"2024-04-26T20:01:07.421059+0000","last_fullsized":"2024-04-26T20:01:07.421059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929118+0000","last_change":"2024-04-26T20:01:05.943085+0000","last_active":"2024-04-26T20:01:06.929118+0000","last_peered":"2024-04-26T20:01:06.929118+0000","last_clean":"2024-04-26T20:01:06.929118+0000","last_became_active":"2024-04-26T20:01:05.940081+0000","last_became_peered":"2024-04-26T20:01:05.940081+0000","last_unstale":"2024-04-26T20:01:06.929118+0000","last_undegraded":"2024-04-26T20:01:06.929118+0000","last_fullsized":"2024-04-26T20:01:06.929118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:22.045959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928666+0000","last_change":"2024-04-26T20:01:05.932062+0000","last_active":"2024-04-26T20:01:06.928666+0000","last_peered":"2024-04-26T20:01:06.928666+0000","last_clean":"2024-04-26T20:01:06.928666+0000","last_became_active":"2024-04-26T20:01:05.931734+0000","last_became_peered":"2024-04-26T20:01:05.931734+0000","last_unstale":"2024-04-26T20:01:06.928666+0000","last_undegraded":"2024-04-26T20:01:06.928666+0000","last_fullsized":"2024-04-26T20:01:06.928666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928891+0000","last_change":"2024-04-26T20:01:05.926555+0000","last_active":"2024-04-26T20:01:06.928891+0000","last_peered":"2024-04-26T20:01:06.928891+0000","last_clean":"2024-04-26T20:01:06.928891+0000","last_became_active":"2024-04-26T20:01:05.925941+0000","last_became_peered":"2024-04-26T20:01:05.925941+0000","last_unstale":"2024-04-26T20:01:06.928891+0000","last_undegraded":"2024-04-26T20:01:06.928891+0000","last_fullsized":"2024-04-26T20:01:06.928891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:44.520959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934038+0000","last_change":"2024-04-26T20:01:05.935796+0000","last_active":"2024-04-26T20:01:06.934038+0000","last_peered":"2024-04-26T20:01:06.934038+0000","last_clean":"2024-04-26T20:01:06.934038+0000","last_became_active":"2024-04-26T20:01:05.935441+0000","last_became_peered":"2024-04-26T20:01:05.935441+0000","last_unstale":"2024-04-26T20:01:06.934038+0000","last_undegraded":"2024-04-26T20:01:06.934038+0000","last_fullsized":"2024-04-26T20:01:06.934038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420885+0000","last_change":"2024-04-26T20:01:05.937693+0000","last_active":"2024-04-26T20:01:07.420885+0000","last_peered":"2024-04-26T20:01:07.420885+0000","last_clean":"2024-04-26T20:01:07.420885+0000","last_became_active":"2024-04-26T20:01:05.937313+0000","last_became_peered":"2024-04-26T20:01:05.937313+0000","last_unstale":"2024-04-26T20:01:07.420885+0000","last_undegraded":"2024-04-26T20:01:07.420885+0000","last_fullsized":"2024-04-26T20:01:07.420885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932008+0000","last_change":"2024-04-26T20:01:05.932177+0000","last_active":"2024-04-26T20:01:06.932008+0000","last_peered":"2024-04-26T20:01:06.932008+0000","last_clean":"2024-04-26T20:01:06.932008+0000","last_became_active":"2024-04-26T20:01:05.931858+0000","last_became_peered":"2024-04-26T20:01:05.931858+0000","last_unstale":"2024-04-26T20:01:06.932008+0000","last_undegraded":"2024-04-26T20:01:06.932008+0000","last_fullsized":"2024-04-26T20:01:06.932008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421612+0000","last_change":"2024-04-26T20:01:05.925756+0000","last_active":"2024-04-26T20:01:07.421612+0000","last_peered":"2024-04-26T20:01:07.421612+0000","last_clean":"2024-04-26T20:01:07.421612+0000","last_became_active":"2024-04-26T20:01:05.925478+0000","last_became_peered":"2024-04-26T20:01:05.925478+0000","last_unstale":"2024-04-26T20:01:07.421612+0000","last_undegraded":"2024-04-26T20:01:07.421612+0000","last_fullsized":"2024-04-26T20:01:07.421612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:59.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932258+0000","last_change":"2024-04-26T20:01:05.934504+0000","last_active":"2024-04-26T20:01:06.932258+0000","last_peered":"2024-04-26T20:01:06.932258+0000","last_clean":"2024-04-26T20:01:06.932258+0000","last_became_active":"2024-04-26T20:01:05.934031+0000","last_became_peered":"2024-04-26T20:01:05.934031+0000","last_unstale":"2024-04-26T20:01:06.932258+0000","last_undegraded":"2024-04-26T20:01:06.932258+0000","last_fullsized":"2024-04-26T20:01:06.932258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424485+0000","last_change":"2024-04-26T20:01:05.931477+0000","last_active":"2024-04-26T20:01:07.424485+0000","last_peered":"2024-04-26T20:01:07.424485+0000","last_clean":"2024-04-26T20:01:07.424485+0000","last_became_active":"2024-04-26T20:01:05.930780+0000","last_became_peered":"2024-04-26T20:01:05.930780+0000","last_unstale":"2024-04-26T20:01:07.424485+0000","last_undegraded":"2024-04-26T20:01:07.424485+0000","last_fullsized":"2024-04-26T20:01:07.424485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928987+0000","last_change":"2024-04-26T20:01:05.921417+0000","last_active":"2024-04-26T20:01:06.928987+0000","last_peered":"2024-04-26T20:01:06.928987+0000","last_clean":"2024-04-26T20:01:06.928987+0000","last_became_active":"2024-04-26T20:01:05.920831+0000","last_became_peered":"2024-04-26T20:01:05.920831+0000","last_unstale":"2024-04-26T20:01:06.928987+0000","last_undegraded":"2024-04-26T20:01:06.928987+0000","last_fullsized":"2024-04-26T20:01:06.928987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:42.461439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424420+0000","last_change":"2024-04-26T20:01:05.930517+0000","last_active":"2024-04-26T20:01:07.424420+0000","last_peered":"2024-04-26T20:01:07.424420+0000","last_clean":"2024-04-26T20:01:07.424420+0000","last_became_active":"2024-04-26T20:01:05.929798+0000","last_became_peered":"2024-04-26T20:01:05.929798+0000","last_unstale":"2024-04-26T20:01:07.424420+0000","last_undegraded":"2024-04-26T20:01:07.424420+0000","last_fullsized":"2024-04-26T20:01:07.424420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425412+0000","last_change":"2024-04-26T20:01:05.929755+0000","last_active":"2024-04-26T20:01:07.425412+0000","last_peered":"2024-04-26T20:01:07.425412+0000","last_clean":"2024-04-26T20:01:07.425412+0000","last_became_active":"2024-04-26T20:01:05.929417+0000","last_became_peered":"2024-04-26T20:01:05.929417+0000","last_unstale":"2024-04-26T20:01:07.425412+0000","last_undegraded":"2024-04-26T20:01:07.425412+0000","last_fullsized":"2024-04-26T20:01:07.425412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422170+0000","last_change":"2024-04-26T20:01:05.928552+0000","last_active":"2024-04-26T20:01:07.422170+0000","last_peered":"2024-04-26T20:01:07.422170+0000","last_clean":"2024-04-26T20:01:07.422170+0000","last_became_active":"2024-04-26T20:01:05.928021+0000","last_became_peered":"2024-04-26T20:01:05.928021+0000","last_unstale":"2024-04-26T20:01:07.422170+0000","last_undegraded":"2024-04-26T20:01:07.422170+0000","last_fullsized":"2024-04-26T20:01:07.422170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934368+0000","last_change":"2024-04-26T20:01:05.936603+0000","last_active":"2024-04-26T20:01:06.934368+0000","last_peered":"2024-04-26T20:01:06.934368+0000","last_clean":"2024-04-26T20:01:06.934368+0000","last_became_active":"2024-04-26T20:01:05.936316+0000","last_became_peered":"2024-04-26T20:01:05.936316+0000","last_unstale":"2024-04-26T20:01:06.934368+0000","last_undegraded":"2024-04-26T20:01:06.934368+0000","last_fullsized":"2024-04-26T20:01:06.934368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423191+0000","last_change":"2024-04-26T20:01:04.900074+0000","last_active":"2024-04-26T20:01:07.423191+0000","last_peered":"2024-04-26T20:01:07.423191+0000","last_clean":"2024-04-26T20:01:07.423191+0000","last_became_active":"2024-04-26T20:01:04.898252+0000","last_became_peered":"2024-04-26T20:01:04.898252+0000","last_unstale":"2024-04-26T20:01:07.423191+0000","last_undegraded":"2024-04-26T20:01:07.423191+0000","last_fullsized":"2024-04-26T20:01:07.423191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422945+0000","last_change":"2024-04-26T20:01:05.922935+0000","last_active":"2024-04-26T20:01:07.422945+0000","last_peered":"2024-04-26T20:01:07.422945+0000","last_clean":"2024-04-26T20:01:07.422945+0000","last_became_active":"2024-04-26T20:01:05.922482+0000","last_became_peered":"2024-04-26T20:01:05.922482+0000","last_unstale":"2024-04-26T20:01:07.422945+0000","last_undegraded":"2024-04-26T20:01:07.422945+0000","last_fullsized":"2024-04-26T20:01:07.422945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934296+0000","last_change":"2024-04-26T20:01:04.902720+0000","last_active":"2024-04-26T20:01:06.934296+0000","last_peered":"2024-04-26T20:01:06.934296+0000","last_clean":"2024-04-26T20:01:06.934296+0000","last_became_active":"2024-04-26T20:01:04.902158+0000","last_became_peered":"2024-04-26T20:01:04.902158+0000","last_unstale":"2024-04-26T20:01:06.934296+0000","last_undegraded":"2024-04-26T20:01:06.934296+0000","last_fullsized":"2024-04-26T20:01:06.934296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928986+0000","last_change":"2024-04-26T20:01:05.926959+0000","last_active":"2024-04-26T20:01:06.928986+0000","last_peered":"2024-04-26T20:01:06.928986+0000","last_clean":"2024-04-26T20:01:06.928986+0000","last_became_active":"2024-04-26T20:01:05.926614+0000","last_became_peered":"2024-04-26T20:01:05.926614+0000","last_unstale":"2024-04-26T20:01:06.928986+0000","last_undegraded":"2024-04-26T20:01:06.928986+0000","last_fullsized":"2024-04-26T20:01:06.928986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420679+0000","last_change":"2024-04-26T20:01:04.897236+0000","last_active":"2024-04-26T20:01:07.420679+0000","last_peered":"2024-04-26T20:01:07.420679+0000","last_clean":"2024-04-26T20:01:07.420679+0000","last_became_active":"2024-04-26T20:01:04.896622+0000","last_became_peered":"2024-04-26T20:01:04.896622+0000","last_unstale":"2024-04-26T20:01:07.420679+0000","last_undegraded":"2024-04-26T20:01:07.420679+0000","last_fullsized":"2024-04-26T20:01:07.420679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:43.508853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932354+0000","last_change":"2024-04-26T20:01:05.934290+0000","last_active":"2024-04-26T20:01:06.932354+0000","last_peered":"2024-04-26T20:01:06.932354+0000","last_clean":"2024-04-26T20:01:06.932354+0000","last_became_active":"2024-04-26T20:01:05.933188+0000","last_became_peered":"2024-04-26T20:01:05.933188+0000","last_unstale":"2024-04-26T20:01:06.932354+0000","last_undegraded":"2024-04-26T20:01:06.932354+0000","last_fullsized":"2024-04-26T20:01:06.932354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929631+0000","last_change":"2024-04-26T20:01:04.879892+0000","last_active":"2024-04-26T20:01:06.929631+0000","last_peered":"2024-04-26T20:01:06.929631+0000","last_clean":"2024-04-26T20:01:06.929631+0000","last_became_active":"2024-04-26T20:01:04.879459+0000","last_became_peered":"2024-04-26T20:01:04.879459+0000","last_unstale":"2024-04-26T20:01:06.929631+0000","last_undegraded":"2024-04-26T20:01:06.929631+0000","last_fullsized":"2024-04-26T20:01:06.929631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423962+0000","last_change":"2024-04-26T20:01:05.930771+0000","last_active":"2024-04-26T20:01:07.423962+0000","last_peered":"2024-04-26T20:01:07.423962+0000","last_clean":"2024-04-26T20:01:07.423962+0000","last_became_active":"2024-04-26T20:01:05.930210+0000","last_became_peered":"2024-04-26T20:01:05.930210+0000","last_unstale":"2024-04-26T20:01:07.423962+0000","last_undegraded":"2024-04-26T20:01:07.423962+0000","last_fullsized":"2024-04-26T20:01:07.423962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929116+0000","last_change":"2024-04-26T20:01:04.899358+0000","last_active":"2024-04-26T20:01:06.929116+0000","last_peered":"2024-04-26T20:01:06.929116+0000","last_clean":"2024-04-26T20:01:06.929116+0000","last_became_active":"2024-04-26T20:01:04.899066+0000","last_became_peered":"2024-04-26T20:01:04.899066+0000","last_unstale":"2024-04-26T20:01:06.929116+0000","last_undegraded":"2024-04-26T20:01:06.929116+0000","last_fullsized":"2024-04-26T20:01:06.929116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421435+0000","last_change":"2024-04-26T20:01:05.925719+0000","last_active":"2024-04-26T20:01:07.421435+0000","last_peered":"2024-04-26T20:01:07.421435+0000","last_clean":"2024-04-26T20:01:07.421435+0000","last_became_active":"2024-04-26T20:01:05.925374+0000","last_became_peered":"2024-04-26T20:01:05.925374+0000","last_unstale":"2024-04-26T20:01:07.421435+0000","last_undegraded":"2024-04-26T20:01:07.421435+0000","last_fullsized":"2024-04-26T20:01:07.421435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929176+0000","last_change":"2024-04-26T20:01:04.896700+0000","last_active":"2024-04-26T20:01:06.929176+0000","last_peered":"2024-04-26T20:01:06.929176+0000","last_clean":"2024-04-26T20:01:06.929176+0000","last_became_active":"2024-04-26T20:01:04.896384+0000","last_became_peered":"2024-04-26T20:01:04.896384+0000","last_unstale":"2024-04-26T20:01:06.929176+0000","last_undegraded":"2024-04-26T20:01:06.929176+0000","last_fullsized":"2024-04-26T20:01:06.929176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:07.281520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928528+0000","last_change":"2024-04-26T20:01:05.928751+0000","last_active":"2024-04-26T20:01:06.928528+0000","last_peered":"2024-04-26T20:01:06.928528+0000","last_clean":"2024-04-26T20:01:06.928528+0000","last_became_active":"2024-04-26T20:01:05.928443+0000","last_became_peered":"2024-04-26T20:01:05.928443+0000","last_unstale":"2024-04-26T20:01:06.928528+0000","last_undegraded":"2024-04-26T20:01:06.928528+0000","last_fullsized":"2024-04-26T20:01:06.928528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420440+0000","last_change":"2024-04-26T20:01:04.891168+0000","last_active":"2024-04-26T20:01:07.420440+0000","last_peered":"2024-04-26T20:01:07.420440+0000","last_clean":"2024-04-26T20:01:07.420440+0000","last_became_active":"2024-04-26T20:01:04.890897+0000","last_became_peered":"2024-04-26T20:01:04.890897+0000","last_unstale":"2024-04-26T20:01:07.420440+0000","last_undegraded":"2024-04-26T20:01:07.420440+0000","last_fullsized":"2024-04-26T20:01:07.420440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:38.348659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928313+0000","last_change":"2024-04-26T20:01:05.918385+0000","last_active":"2024-04-26T20:01:06.928313+0000","last_peered":"2024-04-26T20:01:06.928313+0000","last_clean":"2024-04-26T20:01:06.928313+0000","last_became_active":"2024-04-26T20:01:05.918117+0000","last_became_peered":"2024-04-26T20:01:05.918117+0000","last_unstale":"2024-04-26T20:01:06.928313+0000","last_undegraded":"2024-04-26T20:01:06.928313+0000","last_fullsized":"2024-04-26T20:01:06.928313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928076+0000","last_change":"2024-04-26T20:01:04.900028+0000","last_active":"2024-04-26T20:01:06.928076+0000","last_peered":"2024-04-26T20:01:06.928076+0000","last_clean":"2024-04-26T20:01:06.928076+0000","last_became_active":"2024-04-26T20:01:04.899740+0000","last_became_peered":"2024-04-26T20:01:04.899740+0000","last_unstale":"2024-04-26T20:01:06.928076+0000","last_undegraded":"2024-04-26T20:01:06.928076+0000","last_fullsized":"2024-04-26T20:01:06.928076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424997+0000","last_change":"2024-04-26T20:01:05.924827+0000","last_active":"2024-04-26T20:01:07.424997+0000","last_peered":"2024-04-26T20:01:07.424997+0000","last_clean":"2024-04-26T20:01:07.424997+0000","last_became_active":"2024-04-26T20:01:05.924343+0000","last_became_peered":"2024-04-26T20:01:05.924343+0000","last_unstale":"2024-04-26T20:01:07.424997+0000","last_undegraded":"2024-04-26T20:01:07.424997+0000","last_fullsized":"2024-04-26T20:01:07.424997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421656+0000","last_change":"2024-04-26T20:01:04.904429+0000","last_active":"2024-04-26T20:01:07.421656+0000","last_peered":"2024-04-26T20:01:07.421656+0000","last_clean":"2024-04-26T20:01:07.421656+0000","last_became_active":"2024-04-26T20:01:04.904127+0000","last_became_peered":"2024-04-26T20:01:04.904127+0000","last_unstale":"2024-04-26T20:01:07.421656+0000","last_undegraded":"2024-04-26T20:01:07.421656+0000","last_fullsized":"2024-04-26T20:01:07.421656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:41.449334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421187+0000","last_change":"2024-04-26T20:01:05.930979+0000","last_active":"2024-04-26T20:01:07.421187+0000","last_peered":"2024-04-26T20:01:07.421187+0000","last_clean":"2024-04-26T20:01:07.421187+0000","last_became_active":"2024-04-26T20:01:05.930695+0000","last_became_peered":"2024-04-26T20:01:05.930695+0000","last_unstale":"2024-04-26T20:01:07.421187+0000","last_undegraded":"2024-04-26T20:01:07.421187+0000","last_fullsized":"2024-04-26T20:01:07.421187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:53.308900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934435+0000","last_change":"2024-04-26T20:01:04.902692+0000","last_active":"2024-04-26T20:01:06.934435+0000","last_peered":"2024-04-26T20:01:06.934435+0000","last_clean":"2024-04-26T20:01:06.934435+0000","last_became_active":"2024-04-26T20:01:04.901817+0000","last_became_peered":"2024-04-26T20:01:04.901817+0000","last_unstale":"2024-04-26T20:01:06.934435+0000","last_undegraded":"2024-04-26T20:01:06.934435+0000","last_fullsized":"2024-04-26T20:01:06.934435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421065+0000","last_change":"2024-04-26T20:01:05.922655+0000","last_active":"2024-04-26T20:01:07.421065+0000","last_peered":"2024-04-26T20:01:07.421065+0000","last_clean":"2024-04-26T20:01:07.421065+0000","last_became_active":"2024-04-26T20:01:05.919065+0000","last_became_peered":"2024-04-26T20:01:05.919065+0000","last_unstale":"2024-04-26T20:01:07.421065+0000","last_undegraded":"2024-04-26T20:01:07.421065+0000","last_fullsized":"2024-04-26T20:01:07.421065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423049+0000","last_change":"2024-04-26T20:01:04.903315+0000","last_active":"2024-04-26T20:01:07.423049+0000","last_peered":"2024-04-26T20:01:07.423049+0000","last_clean":"2024-04-26T20:01:07.423049+0000","last_became_active":"2024-04-26T20:01:04.902750+0000","last_became_peered":"2024-04-26T20:01:04.902750+0000","last_unstale":"2024-04-26T20:01:07.423049+0000","last_undegraded":"2024-04-26T20:01:07.423049+0000","last_fullsized":"2024-04-26T20:01:07.423049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422336+0000","last_change":"2024-04-26T20:01:05.926523+0000","last_active":"2024-04-26T20:01:07.422336+0000","last_peered":"2024-04-26T20:01:07.422336+0000","last_clean":"2024-04-26T20:01:07.422336+0000","last_became_active":"2024-04-26T20:01:05.924748+0000","last_became_peered":"2024-04-26T20:01:05.924748+0000","last_unstale":"2024-04-26T20:01:07.422336+0000","last_undegraded":"2024-04-26T20:01:07.422336+0000","last_fullsized":"2024-04-26T20:01:07.422336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421419+0000","last_change":"2024-04-26T20:01:04.896296+0000","last_active":"2024-04-26T20:01:07.421419+0000","last_peered":"2024-04-26T20:01:07.421419+0000","last_clean":"2024-04-26T20:01:07.421419+0000","last_became_active":"2024-04-26T20:01:04.896034+0000","last_became_peered":"2024-04-26T20:01:04.896034+0000","last_unstale":"2024-04-26T20:01:07.421419+0000","last_undegraded":"2024-04-26T20:01:07.421419+0000","last_fullsized":"2024-04-26T20:01:07.421419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.259984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929084+0000","last_change":"2024-04-26T20:01:05.924273+0000","last_active":"2024-04-26T20:01:06.929084+0000","last_peered":"2024-04-26T20:01:06.929084+0000","last_clean":"2024-04-26T20:01:06.929084+0000","last_became_active":"2024-04-26T20:01:05.924002+0000","last_became_peered":"2024-04-26T20:01:05.924002+0000","last_unstale":"2024-04-26T20:01:06.929084+0000","last_undegraded":"2024-04-26T20:01:06.929084+0000","last_fullsized":"2024-04-26T20:01:06.929084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929280+0000","last_change":"2024-04-26T20:01:04.902304+0000","last_active":"2024-04-26T20:01:06.929280+0000","last_peered":"2024-04-26T20:01:06.929280+0000","last_clean":"2024-04-26T20:01:06.929280+0000","last_became_active":"2024-04-26T20:01:04.901986+0000","last_became_peered":"2024-04-26T20:01:04.901986+0000","last_unstale":"2024-04-26T20:01:06.929280+0000","last_undegraded":"2024-04-26T20:01:06.929280+0000","last_fullsized":"2024-04-26T20:01:06.929280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:19.030356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421032+0000","last_change":"2024-04-26T20:01:05.925014+0000","last_active":"2024-04-26T20:01:07.421032+0000","last_peered":"2024-04-26T20:01:07.421032+0000","last_clean":"2024-04-26T20:01:07.421032+0000","last_became_active":"2024-04-26T20:01:05.924734+0000","last_became_peered":"2024-04-26T20:01:05.924734+0000","last_unstale":"2024-04-26T20:01:07.421032+0000","last_undegraded":"2024-04-26T20:01:07.421032+0000","last_fullsized":"2024-04-26T20:01:07.421032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929907+0000","last_change":"2024-04-26T20:01:04.898231+0000","last_active":"2024-04-26T20:01:06.929907+0000","last_peered":"2024-04-26T20:01:06.929907+0000","last_clean":"2024-04-26T20:01:06.929907+0000","last_became_active":"2024-04-26T20:01:04.896868+0000","last_became_peered":"2024-04-26T20:01:04.896868+0000","last_unstale":"2024-04-26T20:01:06.929907+0000","last_undegraded":"2024-04-26T20:01:06.929907+0000","last_fullsized":"2024-04-26T20:01:06.929907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422559+0000","last_change":"2024-04-26T20:01:05.928317+0000","last_active":"2024-04-26T20:01:07.422559+0000","last_peered":"2024-04-26T20:01:07.422559+0000","last_clean":"2024-04-26T20:01:07.422559+0000","last_became_active":"2024-04-26T20:01:05.928030+0000","last_became_peered":"2024-04-26T20:01:05.928030+0000","last_unstale":"2024-04-26T20:01:07.422559+0000","last_undegraded":"2024-04-26T20:01:07.422559+0000","last_fullsized":"2024-04-26T20:01:07.422559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420224+0000","last_change":"2024-04-26T20:01:04.904256+0000","last_active":"2024-04-26T20:01:07.420224+0000","last_peered":"2024-04-26T20:01:07.420224+0000","last_clean":"2024-04-26T20:01:07.420224+0000","last_became_active":"2024-04-26T20:01:04.903998+0000","last_became_peered":"2024-04-26T20:01:04.903998+0000","last_unstale":"2024-04-26T20:01:07.420224+0000","last_undegraded":"2024-04-26T20:01:07.420224+0000","last_fullsized":"2024-04-26T20:01:07.420224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425586+0000","last_change":"2024-04-26T20:01:05.919282+0000","last_active":"2024-04-26T20:01:07.425586+0000","last_peered":"2024-04-26T20:01:07.425586+0000","last_clean":"2024-04-26T20:01:07.425586+0000","last_became_active":"2024-04-26T20:01:05.918943+0000","last_became_peered":"2024-04-26T20:01:05.918943+0000","last_unstale":"2024-04-26T20:01:07.425586+0000","last_undegraded":"2024-04-26T20:01:07.425586+0000","last_fullsized":"2024-04-26T20:01:07.425586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421590+0000","last_change":"2024-04-26T20:01:04.894197+0000","last_active":"2024-04-26T20:01:07.421590+0000","last_peered":"2024-04-26T20:01:07.421590+0000","last_clean":"2024-04-26T20:01:07.421590+0000","last_became_active":"2024-04-26T20:01:04.893919+0000","last_became_peered":"2024-04-26T20:01:04.893919+0000","last_unstale":"2024-04-26T20:01:07.421590+0000","last_undegraded":"2024-04-26T20:01:07.421590+0000","last_fullsized":"2024-04-26T20:01:07.421590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:31.300172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928720+0000","last_change":"2024-04-26T20:01:05.922973+0000","last_active":"2024-04-26T20:01:06.928720+0000","last_peered":"2024-04-26T20:01:06.928720+0000","last_clean":"2024-04-26T20:01:06.928720+0000","last_became_active":"2024-04-26T20:01:05.922657+0000","last_became_peered":"2024-04-26T20:01:05.922657+0000","last_unstale":"2024-04-26T20:01:06.928720+0000","last_undegraded":"2024-04-26T20:01:06.928720+0000","last_fullsized":"2024-04-26T20:01:06.928720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932134+0000","last_change":"2024-04-26T20:01:04.898895+0000","last_active":"2024-04-26T20:01:06.932134+0000","last_peered":"2024-04-26T20:01:06.932134+0000","last_clean":"2024-04-26T20:01:06.932134+0000","last_became_active":"2024-04-26T20:01:04.898535+0000","last_became_peered":"2024-04-26T20:01:04.898535+0000","last_unstale":"2024-04-26T20:01:06.932134+0000","last_undegraded":"2024-04-26T20:01:06.932134+0000","last_fullsized":"2024-04-26T20:01:06.932134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422713+0000","last_change":"2024-04-26T20:01:05.927162+0000","last_active":"2024-04-26T20:01:07.422713+0000","last_peered":"2024-04-26T20:01:07.422713+0000","last_clean":"2024-04-26T20:01:07.422713+0000","last_became_active":"2024-04-26T20:01:05.926785+0000","last_became_peered":"2024-04-26T20:01:05.926785+0000","last_unstale":"2024-04-26T20:01:07.422713+0000","last_undegraded":"2024-04-26T20:01:07.422713+0000","last_fullsized":"2024-04-26T20:01:07.422713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929242+0000","last_change":"2024-04-26T20:01:04.902861+0000","last_active":"2024-04-26T20:01:06.929242+0000","last_peered":"2024-04-26T20:01:06.929242+0000","last_clean":"2024-04-26T20:01:06.929242+0000","last_became_active":"2024-04-26T20:01:04.902393+0000","last_became_peered":"2024-04-26T20:01:04.902393+0000","last_unstale":"2024-04-26T20:01:06.929242+0000","last_undegraded":"2024-04-26T20:01:06.929242+0000","last_fullsized":"2024-04-26T20:01:06.929242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:26.049252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931248+0000","last_change":"2024-04-26T20:01:05.930171+0000","last_active":"2024-04-26T20:01:06.931248+0000","last_peered":"2024-04-26T20:01:06.931248+0000","last_clean":"2024-04-26T20:01:06.931248+0000","last_became_active":"2024-04-26T20:01:05.929737+0000","last_became_peered":"2024-04-26T20:01:05.929737+0000","last_unstale":"2024-04-26T20:01:06.931248+0000","last_undegraded":"2024-04-26T20:01:06.931248+0000","last_fullsized":"2024-04-26T20:01:06.931248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929103+0000","last_change":"2024-04-26T20:01:04.887577+0000","last_active":"2024-04-26T20:01:06.929103+0000","last_peered":"2024-04-26T20:01:06.929103+0000","last_clean":"2024-04-26T20:01:06.929103+0000","last_became_active":"2024-04-26T20:01:04.887059+0000","last_became_peered":"2024-04-26T20:01:04.887059+0000","last_unstale":"2024-04-26T20:01:06.929103+0000","last_undegraded":"2024-04-26T20:01:06.929103+0000","last_fullsized":"2024-04-26T20:01:06.929103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929299+0000","last_change":"2024-04-26T20:01:05.927774+0000","last_active":"2024-04-26T20:01:06.929299+0000","last_peered":"2024-04-26T20:01:06.929299+0000","last_clean":"2024-04-26T20:01:06.929299+0000","last_became_active":"2024-04-26T20:01:05.927459+0000","last_became_peered":"2024-04-26T20:01:05.927459+0000","last_unstale":"2024-04-26T20:01:06.929299+0000","last_undegraded":"2024-04-26T20:01:06.929299+0000","last_fullsized":"2024-04-26T20:01:06.929299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:07.867885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928642+0000","last_change":"2024-04-26T20:01:04.897364+0000","last_active":"2024-04-26T20:01:06.928642+0000","last_peered":"2024-04-26T20:01:06.928642+0000","last_clean":"2024-04-26T20:01:06.928642+0000","last_became_active":"2024-04-26T20:01:04.896186+0000","last_became_peered":"2024-04-26T20:01:04.896186+0000","last_unstale":"2024-04-26T20:01:06.928642+0000","last_undegraded":"2024-04-26T20:01:06.928642+0000","last_fullsized":"2024-04-26T20:01:06.928642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928890+0000","last_change":"2024-04-26T20:01:05.927806+0000","last_active":"2024-04-26T20:01:06.928890+0000","last_peered":"2024-04-26T20:01:06.928890+0000","last_clean":"2024-04-26T20:01:06.928890+0000","last_became_active":"2024-04-26T20:01:05.927383+0000","last_became_peered":"2024-04-26T20:01:05.927383+0000","last_unstale":"2024-04-26T20:01:06.928890+0000","last_undegraded":"2024-04-26T20:01:06.928890+0000","last_fullsized":"2024-04-26T20:01:06.928890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:26.965346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.933989+0000","last_change":"2024-04-26T20:01:04.889881+0000","last_active":"2024-04-26T20:01:06.933989+0000","last_peered":"2024-04-26T20:01:06.933989+0000","last_clean":"2024-04-26T20:01:06.933989+0000","last_became_active":"2024-04-26T20:01:04.889623+0000","last_became_peered":"2024-04-26T20:01:04.889623+0000","last_unstale":"2024-04-26T20:01:06.933989+0000","last_undegraded":"2024-04-26T20:01:06.933989+0000","last_fullsized":"2024-04-26T20:01:06.933989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929270+0000","last_change":"2024-04-26T20:01:05.935015+0000","last_active":"2024-04-26T20:01:06.929270+0000","last_peered":"2024-04-26T20:01:06.929270+0000","last_clean":"2024-04-26T20:01:06.929270+0000","last_became_active":"2024-04-26T20:01:05.934164+0000","last_became_peered":"2024-04-26T20:01:05.934164+0000","last_unstale":"2024-04-26T20:01:06.929270+0000","last_undegraded":"2024-04-26T20:01:06.929270+0000","last_fullsized":"2024-04-26T20:01:06.929270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:10.565596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422516+0000","last_change":"2024-04-26T20:01:04.901055+0000","last_active":"2024-04-26T20:01:07.422516+0000","last_peered":"2024-04-26T20:01:07.422516+0000","last_clean":"2024-04-26T20:01:07.422516+0000","last_became_active":"2024-04-26T20:01:04.900083+0000","last_became_peered":"2024-04-26T20:01:04.900083+0000","last_unstale":"2024-04-26T20:01:07.422516+0000","last_undegraded":"2024-04-26T20:01:07.422516+0000","last_fullsized":"2024-04-26T20:01:07.422516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928234+0000","last_change":"2024-04-26T20:01:05.938285+0000","last_active":"2024-04-26T20:01:06.928234+0000","last_peered":"2024-04-26T20:01:06.928234+0000","last_clean":"2024-04-26T20:01:06.928234+0000","last_became_active":"2024-04-26T20:01:05.934864+0000","last_became_peered":"2024-04-26T20:01:05.934864+0000","last_unstale":"2024-04-26T20:01:06.928234+0000","last_undegraded":"2024-04-26T20:01:06.928234+0000","last_fullsized":"2024-04-26T20:01:06.928234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:08.293626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928147+0000","last_change":"2024-04-26T20:01:04.888976+0000","last_active":"2024-04-26T20:01:06.928147+0000","last_peered":"2024-04-26T20:01:06.928147+0000","last_clean":"2024-04-26T20:01:06.928147+0000","last_became_active":"2024-04-26T20:01:04.888367+0000","last_became_peered":"2024-04-26T20:01:04.888367+0000","last_unstale":"2024-04-26T20:01:06.928147+0000","last_undegraded":"2024-04-26T20:01:06.928147+0000","last_fullsized":"2024-04-26T20:01:06.928147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:51.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421875+0000","last_change":"2024-04-26T20:01:05.922469+0000","last_active":"2024-04-26T20:01:07.421875+0000","last_peered":"2024-04-26T20:01:07.421875+0000","last_clean":"2024-04-26T20:01:07.421875+0000","last_became_active":"2024-04-26T20:01:05.922185+0000","last_became_peered":"2024-04-26T20:01:05.922185+0000","last_unstale":"2024-04-26T20:01:07.421875+0000","last_undegraded":"2024-04-26T20:01:07.421875+0000","last_fullsized":"2024-04-26T20:01:07.421875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420758+0000","last_change":"2024-04-26T20:01:04.906119+0000","last_active":"2024-04-26T20:01:07.420758+0000","last_peered":"2024-04-26T20:01:07.420758+0000","last_clean":"2024-04-26T20:01:07.420758+0000","last_became_active":"2024-04-26T20:01:04.905793+0000","last_became_peered":"2024-04-26T20:01:04.905793+0000","last_unstale":"2024-04-26T20:01:07.420758+0000","last_undegraded":"2024-04-26T20:01:07.420758+0000","last_fullsized":"2024-04-26T20:01:07.420758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928970+0000","last_change":"2024-04-26T20:01:05.928082+0000","last_active":"2024-04-26T20:01:06.928970+0000","last_peered":"2024-04-26T20:01:06.928970+0000","last_clean":"2024-04-26T20:01:06.928970+0000","last_became_active":"2024-04-26T20:01:05.927589+0000","last_became_peered":"2024-04-26T20:01:05.927589+0000","last_unstale":"2024-04-26T20:01:06.928970+0000","last_undegraded":"2024-04-26T20:01:06.928970+0000","last_fullsized":"2024-04-26T20:01:06.928970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:17.326357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932202+0000","last_change":"2024-04-26T20:01:04.897905+0000","last_active":"2024-04-26T20:01:06.932202+0000","last_peered":"2024-04-26T20:01:06.932202+0000","last_clean":"2024-04-26T20:01:06.932202+0000","last_became_active":"2024-04-26T20:01:04.897556+0000","last_became_peered":"2024-04-26T20:01:04.897556+0000","last_unstale":"2024-04-26T20:01:06.932202+0000","last_undegraded":"2024-04-26T20:01:06.932202+0000","last_fullsized":"2024-04-26T20:01:06.932202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424490+0000","last_change":"2024-04-26T20:01:05.926340+0000","last_active":"2024-04-26T20:01:07.424490+0000","last_peered":"2024-04-26T20:01:07.424490+0000","last_clean":"2024-04-26T20:01:07.424490+0000","last_became_active":"2024-04-26T20:01:05.925986+0000","last_became_peered":"2024-04-26T20:01:05.925986+0000","last_unstale":"2024-04-26T20:01:07.424490+0000","last_undegraded":"2024-04-26T20:01:07.424490+0000","last_fullsized":"2024-04-26T20:01:07.424490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934396+0000","last_change":"2024-04-26T20:01:04.888031+0000","last_active":"2024-04-26T20:01:06.934396+0000","last_peered":"2024-04-26T20:01:06.934396+0000","last_clean":"2024-04-26T20:01:06.934396+0000","last_became_active":"2024-04-26T20:01:04.887522+0000","last_became_peered":"2024-04-26T20:01:04.887522+0000","last_unstale":"2024-04-26T20:01:06.934396+0000","last_undegraded":"2024-04-26T20:01:06.934396+0000","last_fullsized":"2024-04-26T20:01:06.934396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934248+0000","last_change":"2024-04-26T20:01:05.939913+0000","last_active":"2024-04-26T20:01:06.934248+0000","last_peered":"2024-04-26T20:01:06.934248+0000","last_clean":"2024-04-26T20:01:06.934248+0000","last_became_active":"2024-04-26T20:01:05.939643+0000","last_became_peered":"2024-04-26T20:01:05.939643+0000","last_unstale":"2024-04-26T20:01:06.934248+0000","last_undegraded":"2024-04-26T20:01:06.934248+0000","last_fullsized":"2024-04-26T20:01:06.934248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931294+0000","last_change":"2024-04-26T20:01:04.882527+0000","last_active":"2024-04-26T20:01:06.931294+0000","last_peered":"2024-04-26T20:01:06.931294+0000","last_clean":"2024-04-26T20:01:06.931294+0000","last_became_active":"2024-04-26T20:01:04.881905+0000","last_became_peered":"2024-04-26T20:01:04.881905+0000","last_unstale":"2024-04-26T20:01:06.931294+0000","last_undegraded":"2024-04-26T20:01:06.931294+0000","last_fullsized":"2024-04-26T20:01:06.931294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422247+0000","last_change":"2024-04-26T20:01:05.926284+0000","last_active":"2024-04-26T20:01:07.422247+0000","last_peered":"2024-04-26T20:01:07.422247+0000","last_clean":"2024-04-26T20:01:07.422247+0000","last_became_active":"2024-04-26T20:01:05.926018+0000","last_became_peered":"2024-04-26T20:01:05.926018+0000","last_unstale":"2024-04-26T20:01:07.422247+0000","last_undegraded":"2024-04-26T20:01:07.422247+0000","last_fullsized":"2024-04-26T20:01:07.422247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:36.109712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934159+0000","last_change":"2024-04-26T20:01:04.898199+0000","last_active":"2024-04-26T20:01:06.934159+0000","last_peered":"2024-04-26T20:01:06.934159+0000","last_clean":"2024-04-26T20:01:06.934159+0000","last_became_active":"2024-04-26T20:01:04.897910+0000","last_became_peered":"2024-04-26T20:01:04.897910+0000","last_unstale":"2024-04-26T20:01:06.934159+0000","last_undegraded":"2024-04-26T20:01:06.934159+0000","last_fullsized":"2024-04-26T20:01:06.934159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934026+0000","last_change":"2024-04-26T20:01:05.931276+0000","last_active":"2024-04-26T20:01:06.934026+0000","last_peered":"2024-04-26T20:01:06.934026+0000","last_clean":"2024-04-26T20:01:06.934026+0000","last_became_active":"2024-04-26T20:01:05.930920+0000","last_became_peered":"2024-04-26T20:01:05.930920+0000","last_unstale":"2024-04-26T20:01:06.934026+0000","last_undegraded":"2024-04-26T20:01:06.934026+0000","last_fullsized":"2024-04-26T20:01:06.934026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930926+0000","last_change":"2024-04-26T20:01:04.898465+0000","last_active":"2024-04-26T20:01:06.930926+0000","last_peered":"2024-04-26T20:01:06.930926+0000","last_clean":"2024-04-26T20:01:06.930926+0000","last_became_active":"2024-04-26T20:01:04.897963+0000","last_became_peered":"2024-04-26T20:01:04.897963+0000","last_unstale":"2024-04-26T20:01:06.930926+0000","last_undegraded":"2024-04-26T20:01:06.930926+0000","last_fullsized":"2024-04-26T20:01:06.930926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420366+0000","last_change":"2024-04-26T20:01:05.914965+0000","last_active":"2024-04-26T20:01:07.420366+0000","last_peered":"2024-04-26T20:01:07.420366+0000","last_clean":"2024-04-26T20:01:07.420366+0000","last_became_active":"2024-04-26T20:01:05.914644+0000","last_became_peered":"2024-04-26T20:01:05.914644+0000","last_unstale":"2024-04-26T20:01:07.420366+0000","last_undegraded":"2024-04-26T20:01:07.420366+0000","last_fullsized":"2024-04-26T20:01:07.420366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928398+0000","last_change":"2024-04-26T20:01:04.894822+0000","last_active":"2024-04-26T20:01:06.928398+0000","last_peered":"2024-04-26T20:01:06.928398+0000","last_clean":"2024-04-26T20:01:06.928398+0000","last_became_active":"2024-04-26T20:01:04.893421+0000","last_became_peered":"2024-04-26T20:01:04.893421+0000","last_unstale":"2024-04-26T20:01:06.928398+0000","last_undegraded":"2024-04-26T20:01:06.928398+0000","last_fullsized":"2024-04-26T20:01:06.928398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:54.615275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928104+0000","last_change":"2024-04-26T20:01:05.924736+0000","last_active":"2024-04-26T20:01:06.928104+0000","last_peered":"2024-04-26T20:01:06.928104+0000","last_clean":"2024-04-26T20:01:06.928104+0000","last_became_active":"2024-04-26T20:01:05.922449+0000","last_became_peered":"2024-04-26T20:01:05.922449+0000","last_unstale":"2024-04-26T20:01:06.928104+0000","last_undegraded":"2024-04-26T20:01:06.928104+0000","last_fullsized":"2024-04-26T20:01:06.928104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420394+0000","last_change":"2024-04-26T20:01:04.904249+0000","last_active":"2024-04-26T20:01:07.420394+0000","last_peered":"2024-04-26T20:01:07.420394+0000","last_clean":"2024-04-26T20:01:07.420394+0000","last_became_active":"2024-04-26T20:01:04.903995+0000","last_became_peered":"2024-04-26T20:01:04.903995+0000","last_unstale":"2024-04-26T20:01:07.420394+0000","last_undegraded":"2024-04-26T20:01:07.420394+0000","last_fullsized":"2024-04-26T20:01:07.420394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424191+0000","last_change":"2024-04-26T20:01:05.938661+0000","last_active":"2024-04-26T20:01:07.424191+0000","last_peered":"2024-04-26T20:01:07.424191+0000","last_clean":"2024-04-26T20:01:07.424191+0000","last_became_active":"2024-04-26T20:01:05.938340+0000","last_became_peered":"2024-04-26T20:01:05.938340+0000","last_unstale":"2024-04-26T20:01:07.424191+0000","last_undegraded":"2024-04-26T20:01:07.424191+0000","last_fullsized":"2024-04-26T20:01:07.424191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421989+0000","last_change":"2024-04-26T20:01:04.903742+0000","last_active":"2024-04-26T20:01:07.421989+0000","last_peered":"2024-04-26T20:01:07.421989+0000","last_clean":"2024-04-26T20:01:07.421989+0000","last_became_active":"2024-04-26T20:01:04.903299+0000","last_became_peered":"2024-04-26T20:01:04.903299+0000","last_unstale":"2024-04-26T20:01:07.421989+0000","last_undegraded":"2024-04-26T20:01:07.421989+0000","last_fullsized":"2024-04-26T20:01:07.421989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423373+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:07.423373+0000","last_peered":"2024-04-26T20:01:07.423373+0000","last_clean":"2024-04-26T20:01:07.423373+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:07.423373+0000","last_undegraded":"2024-04-26T20:01:07.423373+0000","last_fullsized":"2024-04-26T20:01:07.423373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":737138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:01:11.338 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:01:11.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.572+0000 7efe11200640 1 Processor -- start 2024-04-26T20:01:11.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.572+0000 7efe11200640 1 -- start start 2024-04-26T20:01:11.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.573+0000 7efe11200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efe0c0cc410 0x7efe0c0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.573+0000 7efe11200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 0x7efe0c168b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.573+0000 7efe11200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0ca7c0 con 0x7efe0c147d50 2024-04-26T20:01:11.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.573+0000 7efe11200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7efe0c06e680 con 0x7efe0c0cc410 2024-04-26T20:01:11.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.573+0000 7efe11200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efe0c070130 con 0x7efe0c0ca370 2024-04-26T20:01:11.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7efe0c147d50 0x7efe0c0cbd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34360/0 (socket says 172.21.15.88:34360) 2024-04-26T20:01:11.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0b400640 1 -- 172.21.15.88:0/322319676 learned_addr learned my addr 172.21.15.88:0/322319676 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:11.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe03e00640 1 --2- 172.21.15.88:0/322319676 >> v2:172.21.15.88:3300/0 conn(0x7efe0c0cc410 0x7efe0c0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0aa00640 1 -- 172.21.15.88:0/322319676 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2401189189 0 0) 0x7efdfc001110 con 0x7efe0c147d50 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0be00640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 0x7efe0c168b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0aa00640 1 -- 172.21.15.88:0/322319676 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf0003520 con 0x7efe0c147d50 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe03e00640 1 -- 172.21.15.88:0/322319676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 msgr2=0x7efe0c168b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe03e00640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 0x7efe0c168b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe03e00640 1 -- 172.21.15.88:0/322319676 >> v1:172.21.15.88:6789/0 conn(0x7efe0c147d50 legacy=0x7efe0c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe03e00640 1 -- 172.21.15.88:0/322319676 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c0caa40 con 0x7efe0c0cc410 2024-04-26T20:01:11.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.574+0000 7efe0be00640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 0x7efe0c168b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:11.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.575+0000 7efe03e00640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0cc410 0x7efe0c0c9e30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efdf40097a0 tx=0x7efdf402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1525c3f1f02fd14c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe0aa00640 1 -- 172.21.15.88:0/322319676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4037070 con 0x7efe0c0cc410 2024-04-26T20:01:11.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe0aa00640 1 -- 172.21.15.88:0/322319676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf4032440 con 0x7efe0c0cc410 2024-04-26T20:01:11.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe0aa00640 1 -- 172.21.15.88:0/322319676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf40327f0 con 0x7efe0c0cc410 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 -- 172.21.15.88:0/322319676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0cc410 msgr2=0x7efe0c0c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0cc410 0x7efe0c0c9e30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efdf40097a0 tx=0x7efdf402a180 comp rx=0 tx=0).stop 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 -- 172.21.15.88:0/322319676 shutdown_connections 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0ca370 0x7efe0c168b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 --2- 172.21.15.88:0/322319676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0cc410 0x7efe0c0c9e30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 -- 172.21.15.88:0/322319676 >> 172.21.15.88:0/322319676 conn(0x7efe0c0d0ae0 msgr2=0x7efe0c0d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 -- 172.21.15.88:0/322319676 shutdown_connections 2024-04-26T20:01:11.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.576+0000 7efe11200640 1 -- 172.21.15.88:0/322319676 wait complete. 2024-04-26T20:01:11.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.578+0000 7efe11200640 1 Processor -- start 2024-04-26T20:01:11.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.578+0000 7efe11200640 1 -- start start 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0cc410 0x7efe0c0aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe0c147d50 0x7efe0c0ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c06ec00 con 0x7efe0c0ca370 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c06e920 con 0x7efe0c147d50 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe11200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efe0c0ce6b0 con 0x7efe0c0cc410 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe0b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe0b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58206/0 (socket says 172.21.15.88:58206) 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.579+0000 7efe0b400640 1 -- 172.21.15.88:0/3937047956 learned_addr learned my addr 172.21.15.88:0/3937047956 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0be00640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe0c147d50 0x7efe0c0ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 -- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0cc410 msgr2=0x7efe0c0aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0cc410 0x7efe0c0aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 -- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe0c147d50 msgr2=0x7efe0c0ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe0c147d50 0x7efe0c0ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 -- 172.21.15.88:0/3937047956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c0af200 con 0x7efe0c0ca370 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe0b400640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c760 tx=0x7efdfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc017070 con 0x7efe0c0ca370 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc005ad0 con 0x7efe0c0ca370 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc007e20 con 0x7efe0c0ca370 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c0aa650 con 0x7efe0c0ca370 2024-04-26T20:01:11.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.580+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c0aac10 con 0x7efe0c0ca370 2024-04-26T20:01:11.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.581+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe0c0693c0 con 0x7efe0c0ca370 2024-04-26T20:01:11.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.582+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdfc01a070 con 0x7efe0c0ca370 2024-04-26T20:01:11.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.583+0000 7efe03400640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:11.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.583+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efdfc08afb0 con 0x7efe0c0ca370 2024-04-26T20:01:11.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.584+0000 7efe03e00640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:11.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.584+0000 7efe03e00640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 0x7efde806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:11.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.588+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe0c0ca7c0 con 0x7efe0c0ca370 2024-04-26T20:01:11.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.859+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7efe0c076600 con 0x7efe0c0ca370 2024-04-26T20:01:11.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.860+0000 7efe03400640 1 -- 172.21.15.88:0/3937047956 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1013 (secure 0 0 0) 0x7efdfc059340 con 0x7efe0c0ca370 2024-04-26T20:01:11.861 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:11.861 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":110,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714161574,"num_in_osds":8,"osd_in_since":1714161565,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":225628160,"bytes_avail":772868485120,"bytes_total":773094113280,"write_bytes_sec":2350,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4708}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T20:00:52.961814+0000","services":{}},"progress_events":{}} 2024-04-26T20:01:11.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 msgr2=0x7efde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 0x7efde806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efdf4034000 tx=0x7efdf402c000 comp rx=0 tx=0).stop 2024-04-26T20:01:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 msgr2=0x7efe0c1440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00c760 tx=0x7efdfc00cc20 comp rx=0 tx=0).stop 2024-04-26T20:01:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 shutdown_connections 2024-04-26T20:01:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe0c147d50 0x7efe0c0ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efe0c0cc410 0x7efe0c0aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 --2- 172.21.15.88:0/3937047956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe0c0ca370 0x7efe0c1440d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:11.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.863+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 >> 172.21.15.88:0/3937047956 conn(0x7efe0c0d0ae0 msgr2=0x7efe0c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:11.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.864+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 shutdown_connections 2024-04-26T20:01:11.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:11.864+0000 7efe11200640 1 -- 172.21.15.88:0/3937047956 wait complete. 2024-04-26T20:01:11.884 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:01:11.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:01:12.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.083+0000 7f3a6a200640 1 Processor -- start 2024-04-26T20:01:12.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.083+0000 7f3a6a200640 1 -- start start 2024-04-26T20:01:12.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a6a200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3a641598a0 0x7f3a641638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a6a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3a6406c770 0x7f3a6406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a6a200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a640b5fb0 con 0x7f3a6415f740 2024-04-26T20:01:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a6a200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3a64159c80 con 0x7f3a641598a0 2024-04-26T20:01:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a6a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6406d110 con 0x7f3a6406c770 2024-04-26T20:01:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a63e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3a6415f740 0x7f3a6415fb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34362/0 (socket says 172.21.15.88:34362) 2024-04-26T20:01:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a63e00640 1 -- 172.21.15.88:0/3243958795 learned_addr learned my addr 172.21.15.88:0/3243958795 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a63400640 1 --2- 172.21.15.88:0/3243958795 >> v2:172.21.15.88:3300/0 conn(0x7f3a641598a0 0x7f3a641638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a62a00640 1 --2- 172.21.15.88:0/3243958795 >> v2:172.21.15.132:3300/0 conn(0x7f3a6406c770 0x7f3a6406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.085+0000 7f3a63400640 1 -- 172.21.15.88:0/3243958795 >> v2:172.21.15.132:3300/0 conn(0x7f3a6406c770 msgr2=0x7f3a6406cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.086+0000 7f3a63400640 1 --2- 172.21.15.88:0/3243958795 >> v2:172.21.15.132:3300/0 conn(0x7f3a6406c770 0x7f3a6406cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.086+0000 7f3a63400640 1 -- 172.21.15.88:0/3243958795 >> v1:172.21.15.88:6789/0 conn(0x7f3a6415f740 legacy=0x7f3a6415fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.086+0000 7f3a63400640 1 -- 172.21.15.88:0/3243958795 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a64160370 con 0x7f3a641598a0 2024-04-26T20:01:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.086+0000 7f3a62000640 1 -- 172.21.15.88:0/3243958795 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625467475 0 0) 0x7f3a640b5fb0 con 0x7f3a6415f740 2024-04-26T20:01:12.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.087+0000 7f3a63400640 1 --2- 172.21.15.88:0/3243958795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a641598a0 0x7f3a641638b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3a640b3900 tx=0x7f3a5402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c62080d7e97c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:12.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.087+0000 7f3a62000640 1 -- 172.21.15.88:0/3243958795 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54037070 con 0x7f3a641598a0 2024-04-26T20:01:12.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.087+0000 7f3a62000640 1 -- 172.21.15.88:0/3243958795 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a54032440 con 0x7f3a641598a0 2024-04-26T20:01:12.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.087+0000 7f3a62000640 1 -- 172.21.15.88:0/3243958795 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5402ae60 con 0x7f3a641598a0 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 -- 172.21.15.88:0/3243958795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a641598a0 msgr2=0x7f3a641638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 --2- 172.21.15.88:0/3243958795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a641598a0 0x7f3a641638b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3a640b3900 tx=0x7f3a5402a110 comp rx=0 tx=0).stop 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 -- 172.21.15.88:0/3243958795 shutdown_connections 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 --2- 172.21.15.88:0/3243958795 >> v2:172.21.15.132:3300/0 conn(0x7f3a6406c770 0x7f3a6406cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 --2- 172.21.15.88:0/3243958795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a641598a0 0x7f3a641638b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 -- 172.21.15.88:0/3243958795 >> 172.21.15.88:0/3243958795 conn(0x7f3a640b2d50 msgr2=0x7f3a640b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 -- 172.21.15.88:0/3243958795 shutdown_connections 2024-04-26T20:01:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.088+0000 7f3a6a200640 1 -- 172.21.15.88:0/3243958795 wait complete. 2024-04-26T20:01:12.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.089+0000 7f3a6a200640 1 Processor -- start 2024-04-26T20:01:12.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.090+0000 7f3a6a200640 1 -- start start 2024-04-26T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.090+0000 7f3a6a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a6406c770 0x7f3a64158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.090+0000 7f3a6a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a641598a0 0x7f3a6409cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.090+0000 7f3a6a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a6a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64163390 con 0x7f3a6415f740 2024-04-26T20:01:12.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a6a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641630b0 con 0x7f3a641598a0 2024-04-26T20:01:12.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a6a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3a64160650 con 0x7f3a6406c770 2024-04-26T20:01:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a63400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a6406c770 0x7f3a64158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58216/0 (socket says 172.21.15.88:58216) 2024-04-26T20:01:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a63e00640 1 -- 172.21.15.88:0/210136730 learned_addr learned my addr 172.21.15.88:0/210136730 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.091+0000 7f3a62a00640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a641598a0 0x7f3a6409cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 -- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a6406c770 msgr2=0x7f3a64158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a6406c770 0x7f3a64158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 -- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a641598a0 msgr2=0x7f3a6409cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a641598a0 0x7f3a6409cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 -- 172.21.15.88:0/210136730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a641591f0 con 0x7f3a6415f740 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a63e00640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00c780 tx=0x7f3a5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c018070 con 0x7f3a6415f740 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00a640 con 0x7f3a6415f740 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00aab0 con 0x7f3a6415f740 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6409d790 con 0x7f3a6415f740 2024-04-26T20:01:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.092+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6409e030 con 0x7f3a6415f740 2024-04-26T20:01:12.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.094+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64069540 con 0x7f3a6415f740 2024-04-26T20:01:12.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.095+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a5c01b070 con 0x7f3a6415f740 2024-04-26T20:01:12.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.095+0000 7f3a5b400640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 0x7f3a4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:12.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.096+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3a5c08b0c0 con 0x7f3a6415f740 2024-04-26T20:01:12.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.096+0000 7f3a63400640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 0x7f3a4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:12.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.097+0000 7f3a63400640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 0x7f3a4406ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a54034000 tx=0x7f3a5402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:12.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.100+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a5c058e10 con 0x7f3a6415f740 2024-04-26T20:01:12.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.292+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a64073fc0 con 0x7f3a4406c920 2024-04-26T20:01:12.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.303+0000 7f3a5b400640 1 -- 172.21.15.88:0/210136730 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7f3a64073fc0 con 0x7f3a4406c920 2024-04-26T20:01:12.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:12.309 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 msgr2=0x7f3a4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 0x7f3a4406ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a54034000 tx=0x7f3a5402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 msgr2=0x7f3a6409d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c00c780 tx=0x7f3a5c00cc40 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 shutdown_connections 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3a6415f740 0x7f3a6409d180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a641598a0 0x7f3a6409cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3a4406c920 0x7f3a4406ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 --2- 172.21.15.88:0/210136730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3a6406c770 0x7f3a64158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 >> 172.21.15.88:0/210136730 conn(0x7f3a640b2d50 msgr2=0x7f3a640b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 shutdown_connections 2024-04-26T20:01:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:12.308+0000 7f3a6a200640 1 -- 172.21.15.88:0/210136730 wait complete. 2024-04-26T20:01:12.326 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-26T20:01:10.965366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220340,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754380,"statfs":{"total":773094113280,"available":772868485120,"internally_reserved":0,"allocated":4235264,"data_stored":2359716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.099838"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422513+0000","last_change":"2024-04-26T20:01:05.919081+0000","last_active":"2024-04-26T20:01:07.422513+0000","last_peered":"2024-04-26T20:01:07.422513+0000","last_clean":"2024-04-26T20:01:07.422513+0000","last_became_active":"2024-04-26T20:01:05.916236+0000","last_became_peered":"2024-04-26T20:01:05.916236+0000","last_unstale":"2024-04-26T20:01:07.422513+0000","last_undegraded":"2024-04-26T20:01:07.422513+0000","last_fullsized":"2024-04-26T20:01:07.422513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.876764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421185+0000","last_change":"2024-04-26T20:01:05.924948+0000","last_active":"2024-04-26T20:01:07.421185+0000","last_peered":"2024-04-26T20:01:07.421185+0000","last_clean":"2024-04-26T20:01:07.421185+0000","last_became_active":"2024-04-26T20:01:05.924322+0000","last_became_peered":"2024-04-26T20:01:05.924322+0000","last_unstale":"2024-04-26T20:01:07.421185+0000","last_undegraded":"2024-04-26T20:01:07.421185+0000","last_fullsized":"2024-04-26T20:01:07.421185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:39.001253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422789+0000","last_change":"2024-04-26T20:01:05.925708+0000","last_active":"2024-04-26T20:01:07.422789+0000","last_peered":"2024-04-26T20:01:07.422789+0000","last_clean":"2024-04-26T20:01:07.422789+0000","last_became_active":"2024-04-26T20:01:05.924331+0000","last_became_peered":"2024-04-26T20:01:05.924331+0000","last_unstale":"2024-04-26T20:01:07.422789+0000","last_undegraded":"2024-04-26T20:01:07.422789+0000","last_fullsized":"2024-04-26T20:01:07.422789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930927+0000","last_change":"2024-04-26T20:01:05.930430+0000","last_active":"2024-04-26T20:01:06.930927+0000","last_peered":"2024-04-26T20:01:06.930927+0000","last_clean":"2024-04-26T20:01:06.930927+0000","last_became_active":"2024-04-26T20:01:05.930139+0000","last_became_peered":"2024-04-26T20:01:05.930139+0000","last_unstale":"2024-04-26T20:01:06.930927+0000","last_undegraded":"2024-04-26T20:01:06.930927+0000","last_fullsized":"2024-04-26T20:01:06.930927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928615+0000","last_change":"2024-04-26T20:01:05.934500+0000","last_active":"2024-04-26T20:01:06.928615+0000","last_peered":"2024-04-26T20:01:06.928615+0000","last_clean":"2024-04-26T20:01:06.928615+0000","last_became_active":"2024-04-26T20:01:05.933310+0000","last_became_peered":"2024-04-26T20:01:05.933310+0000","last_unstale":"2024-04-26T20:01:06.928615+0000","last_undegraded":"2024-04-26T20:01:06.928615+0000","last_fullsized":"2024-04-26T20:01:06.928615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420434+0000","last_change":"2024-04-26T20:01:05.912926+0000","last_active":"2024-04-26T20:01:07.420434+0000","last_peered":"2024-04-26T20:01:07.420434+0000","last_clean":"2024-04-26T20:01:07.420434+0000","last_became_active":"2024-04-26T20:01:05.912473+0000","last_became_peered":"2024-04-26T20:01:05.912473+0000","last_unstale":"2024-04-26T20:01:07.420434+0000","last_undegraded":"2024-04-26T20:01:07.420434+0000","last_fullsized":"2024-04-26T20:01:07.420434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:51.470568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423817+0000","last_change":"2024-04-26T20:01:05.928943+0000","last_active":"2024-04-26T20:01:07.423817+0000","last_peered":"2024-04-26T20:01:07.423817+0000","last_clean":"2024-04-26T20:01:07.423817+0000","last_became_active":"2024-04-26T20:01:05.928434+0000","last_became_peered":"2024-04-26T20:01:05.928434+0000","last_unstale":"2024-04-26T20:01:07.423817+0000","last_undegraded":"2024-04-26T20:01:07.423817+0000","last_fullsized":"2024-04-26T20:01:07.423817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421731+0000","last_change":"2024-04-26T20:01:05.940280+0000","last_active":"2024-04-26T20:01:07.421731+0000","last_peered":"2024-04-26T20:01:07.421731+0000","last_clean":"2024-04-26T20:01:07.421731+0000","last_became_active":"2024-04-26T20:01:05.940034+0000","last_became_peered":"2024-04-26T20:01:05.940034+0000","last_unstale":"2024-04-26T20:01:07.421731+0000","last_undegraded":"2024-04-26T20:01:07.421731+0000","last_fullsized":"2024-04-26T20:01:07.421731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929681+0000","last_change":"2024-04-26T20:01:05.945356+0000","last_active":"2024-04-26T20:01:06.929681+0000","last_peered":"2024-04-26T20:01:06.929681+0000","last_clean":"2024-04-26T20:01:06.929681+0000","last_became_active":"2024-04-26T20:01:05.945132+0000","last_became_peered":"2024-04-26T20:01:05.945132+0000","last_unstale":"2024-04-26T20:01:06.929681+0000","last_undegraded":"2024-04-26T20:01:06.929681+0000","last_fullsized":"2024-04-26T20:01:06.929681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931906+0000","last_change":"2024-04-26T20:01:05.926545+0000","last_active":"2024-04-26T20:01:06.931906+0000","last_peered":"2024-04-26T20:01:06.931906+0000","last_clean":"2024-04-26T20:01:06.931906+0000","last_became_active":"2024-04-26T20:01:05.924721+0000","last_became_peered":"2024-04-26T20:01:05.924721+0000","last_unstale":"2024-04-26T20:01:06.931906+0000","last_undegraded":"2024-04-26T20:01:06.931906+0000","last_fullsized":"2024-04-26T20:01:06.931906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424805+0000","last_change":"2024-04-26T20:01:05.939937+0000","last_active":"2024-04-26T20:01:07.424805+0000","last_peered":"2024-04-26T20:01:07.424805+0000","last_clean":"2024-04-26T20:01:07.424805+0000","last_became_active":"2024-04-26T20:01:05.939644+0000","last_became_peered":"2024-04-26T20:01:05.939644+0000","last_unstale":"2024-04-26T20:01:07.424805+0000","last_undegraded":"2024-04-26T20:01:07.424805+0000","last_fullsized":"2024-04-26T20:01:07.424805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423650+0000","last_change":"2024-04-26T20:01:05.919891+0000","last_active":"2024-04-26T20:01:07.423650+0000","last_peered":"2024-04-26T20:01:07.423650+0000","last_clean":"2024-04-26T20:01:07.423650+0000","last_became_active":"2024-04-26T20:01:05.919631+0000","last_became_peered":"2024-04-26T20:01:05.919631+0000","last_unstale":"2024-04-26T20:01:07.423650+0000","last_undegraded":"2024-04-26T20:01:07.423650+0000","last_fullsized":"2024-04-26T20:01:07.423650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928767+0000","last_change":"2024-04-26T20:01:05.927158+0000","last_active":"2024-04-26T20:01:06.928767+0000","last_peered":"2024-04-26T20:01:06.928767+0000","last_clean":"2024-04-26T20:01:06.928767+0000","last_became_active":"2024-04-26T20:01:05.926783+0000","last_became_peered":"2024-04-26T20:01:05.926783+0000","last_unstale":"2024-04-26T20:01:06.928767+0000","last_undegraded":"2024-04-26T20:01:06.928767+0000","last_fullsized":"2024-04-26T20:01:06.928767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:01.310367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420598+0000","last_change":"2024-04-26T20:01:05.940579+0000","last_active":"2024-04-26T20:01:07.420598+0000","last_peered":"2024-04-26T20:01:07.420598+0000","last_clean":"2024-04-26T20:01:07.420598+0000","last_became_active":"2024-04-26T20:01:05.940304+0000","last_became_peered":"2024-04-26T20:01:05.940304+0000","last_unstale":"2024-04-26T20:01:07.420598+0000","last_undegraded":"2024-04-26T20:01:07.420598+0000","last_fullsized":"2024-04-26T20:01:07.420598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:21.271559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421597+0000","last_change":"2024-04-26T20:01:05.923735+0000","last_active":"2024-04-26T20:01:07.421597+0000","last_peered":"2024-04-26T20:01:07.421597+0000","last_clean":"2024-04-26T20:01:07.421597+0000","last_became_active":"2024-04-26T20:01:05.923381+0000","last_became_peered":"2024-04-26T20:01:05.923381+0000","last_unstale":"2024-04-26T20:01:07.421597+0000","last_undegraded":"2024-04-26T20:01:07.421597+0000","last_fullsized":"2024-04-26T20:01:07.421597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423136+0000","last_change":"2024-04-26T20:01:05.923995+0000","last_active":"2024-04-26T20:01:07.423136+0000","last_peered":"2024-04-26T20:01:07.423136+0000","last_clean":"2024-04-26T20:01:07.423136+0000","last_became_active":"2024-04-26T20:01:05.923638+0000","last_became_peered":"2024-04-26T20:01:05.923638+0000","last_unstale":"2024-04-26T20:01:07.423136+0000","last_undegraded":"2024-04-26T20:01:07.423136+0000","last_fullsized":"2024-04-26T20:01:07.423136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424734+0000","last_change":"2024-04-26T20:01:05.927247+0000","last_active":"2024-04-26T20:01:07.424734+0000","last_peered":"2024-04-26T20:01:07.424734+0000","last_clean":"2024-04-26T20:01:07.424734+0000","last_became_active":"2024-04-26T20:01:05.924925+0000","last_became_peered":"2024-04-26T20:01:05.924925+0000","last_unstale":"2024-04-26T20:01:07.424734+0000","last_undegraded":"2024-04-26T20:01:07.424734+0000","last_fullsized":"2024-04-26T20:01:07.424734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421254+0000","last_change":"2024-04-26T20:01:05.926554+0000","last_active":"2024-04-26T20:01:07.421254+0000","last_peered":"2024-04-26T20:01:07.421254+0000","last_clean":"2024-04-26T20:01:07.421254+0000","last_became_active":"2024-04-26T20:01:05.926287+0000","last_became_peered":"2024-04-26T20:01:05.926287+0000","last_unstale":"2024-04-26T20:01:07.421254+0000","last_undegraded":"2024-04-26T20:01:07.421254+0000","last_fullsized":"2024-04-26T20:01:07.421254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:43.379844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421059+0000","last_change":"2024-04-26T20:01:05.914996+0000","last_active":"2024-04-26T20:01:07.421059+0000","last_peered":"2024-04-26T20:01:07.421059+0000","last_clean":"2024-04-26T20:01:07.421059+0000","last_became_active":"2024-04-26T20:01:05.914692+0000","last_became_peered":"2024-04-26T20:01:05.914692+0000","last_unstale":"2024-04-26T20:01:07.421059+0000","last_undegraded":"2024-04-26T20:01:07.421059+0000","last_fullsized":"2024-04-26T20:01:07.421059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929118+0000","last_change":"2024-04-26T20:01:05.943085+0000","last_active":"2024-04-26T20:01:06.929118+0000","last_peered":"2024-04-26T20:01:06.929118+0000","last_clean":"2024-04-26T20:01:06.929118+0000","last_became_active":"2024-04-26T20:01:05.940081+0000","last_became_peered":"2024-04-26T20:01:05.940081+0000","last_unstale":"2024-04-26T20:01:06.929118+0000","last_undegraded":"2024-04-26T20:01:06.929118+0000","last_fullsized":"2024-04-26T20:01:06.929118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:22.045959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928666+0000","last_change":"2024-04-26T20:01:05.932062+0000","last_active":"2024-04-26T20:01:06.928666+0000","last_peered":"2024-04-26T20:01:06.928666+0000","last_clean":"2024-04-26T20:01:06.928666+0000","last_became_active":"2024-04-26T20:01:05.931734+0000","last_became_peered":"2024-04-26T20:01:05.931734+0000","last_unstale":"2024-04-26T20:01:06.928666+0000","last_undegraded":"2024-04-26T20:01:06.928666+0000","last_fullsized":"2024-04-26T20:01:06.928666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928891+0000","last_change":"2024-04-26T20:01:05.926555+0000","last_active":"2024-04-26T20:01:06.928891+0000","last_peered":"2024-04-26T20:01:06.928891+0000","last_clean":"2024-04-26T20:01:06.928891+0000","last_became_active":"2024-04-26T20:01:05.925941+0000","last_became_peered":"2024-04-26T20:01:05.925941+0000","last_unstale":"2024-04-26T20:01:06.928891+0000","last_undegraded":"2024-04-26T20:01:06.928891+0000","last_fullsized":"2024-04-26T20:01:06.928891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:44.520959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934038+0000","last_change":"2024-04-26T20:01:05.935796+0000","last_active":"2024-04-26T20:01:06.934038+0000","last_peered":"2024-04-26T20:01:06.934038+0000","last_clean":"2024-04-26T20:01:06.934038+0000","last_became_active":"2024-04-26T20:01:05.935441+0000","last_became_peered":"2024-04-26T20:01:05.935441+0000","last_unstale":"2024-04-26T20:01:06.934038+0000","last_undegraded":"2024-04-26T20:01:06.934038+0000","last_fullsized":"2024-04-26T20:01:06.934038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420885+0000","last_change":"2024-04-26T20:01:05.937693+0000","last_active":"2024-04-26T20:01:07.420885+0000","last_peered":"2024-04-26T20:01:07.420885+0000","last_clean":"2024-04-26T20:01:07.420885+0000","last_became_active":"2024-04-26T20:01:05.937313+0000","last_became_peered":"2024-04-26T20:01:05.937313+0000","last_unstale":"2024-04-26T20:01:07.420885+0000","last_undegraded":"2024-04-26T20:01:07.420885+0000","last_fullsized":"2024-04-26T20:01:07.420885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932008+0000","last_change":"2024-04-26T20:01:05.932177+0000","last_active":"2024-04-26T20:01:06.932008+0000","last_peered":"2024-04-26T20:01:06.932008+0000","last_clean":"2024-04-26T20:01:06.932008+0000","last_became_active":"2024-04-26T20:01:05.931858+0000","last_became_peered":"2024-04-26T20:01:05.931858+0000","last_unstale":"2024-04-26T20:01:06.932008+0000","last_undegraded":"2024-04-26T20:01:06.932008+0000","last_fullsized":"2024-04-26T20:01:06.932008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421612+0000","last_change":"2024-04-26T20:01:05.925756+0000","last_active":"2024-04-26T20:01:07.421612+0000","last_peered":"2024-04-26T20:01:07.421612+0000","last_clean":"2024-04-26T20:01:07.421612+0000","last_became_active":"2024-04-26T20:01:05.925478+0000","last_became_peered":"2024-04-26T20:01:05.925478+0000","last_unstale":"2024-04-26T20:01:07.421612+0000","last_undegraded":"2024-04-26T20:01:07.421612+0000","last_fullsized":"2024-04-26T20:01:07.421612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:59.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932258+0000","last_change":"2024-04-26T20:01:05.934504+0000","last_active":"2024-04-26T20:01:06.932258+0000","last_peered":"2024-04-26T20:01:06.932258+0000","last_clean":"2024-04-26T20:01:06.932258+0000","last_became_active":"2024-04-26T20:01:05.934031+0000","last_became_peered":"2024-04-26T20:01:05.934031+0000","last_unstale":"2024-04-26T20:01:06.932258+0000","last_undegraded":"2024-04-26T20:01:06.932258+0000","last_fullsized":"2024-04-26T20:01:06.932258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424485+0000","last_change":"2024-04-26T20:01:05.931477+0000","last_active":"2024-04-26T20:01:07.424485+0000","last_peered":"2024-04-26T20:01:07.424485+0000","last_clean":"2024-04-26T20:01:07.424485+0000","last_became_active":"2024-04-26T20:01:05.930780+0000","last_became_peered":"2024-04-26T20:01:05.930780+0000","last_unstale":"2024-04-26T20:01:07.424485+0000","last_undegraded":"2024-04-26T20:01:07.424485+0000","last_fullsized":"2024-04-26T20:01:07.424485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928987+0000","last_change":"2024-04-26T20:01:05.921417+0000","last_active":"2024-04-26T20:01:06.928987+0000","last_peered":"2024-04-26T20:01:06.928987+0000","last_clean":"2024-04-26T20:01:06.928987+0000","last_became_active":"2024-04-26T20:01:05.920831+0000","last_became_peered":"2024-04-26T20:01:05.920831+0000","last_unstale":"2024-04-26T20:01:06.928987+0000","last_undegraded":"2024-04-26T20:01:06.928987+0000","last_fullsized":"2024-04-26T20:01:06.928987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:42.461439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424420+0000","last_change":"2024-04-26T20:01:05.930517+0000","last_active":"2024-04-26T20:01:07.424420+0000","last_peered":"2024-04-26T20:01:07.424420+0000","last_clean":"2024-04-26T20:01:07.424420+0000","last_became_active":"2024-04-26T20:01:05.929798+0000","last_became_peered":"2024-04-26T20:01:05.929798+0000","last_unstale":"2024-04-26T20:01:07.424420+0000","last_undegraded":"2024-04-26T20:01:07.424420+0000","last_fullsized":"2024-04-26T20:01:07.424420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425412+0000","last_change":"2024-04-26T20:01:05.929755+0000","last_active":"2024-04-26T20:01:07.425412+0000","last_peered":"2024-04-26T20:01:07.425412+0000","last_clean":"2024-04-26T20:01:07.425412+0000","last_became_active":"2024-04-26T20:01:05.929417+0000","last_became_peered":"2024-04-26T20:01:05.929417+0000","last_unstale":"2024-04-26T20:01:07.425412+0000","last_undegraded":"2024-04-26T20:01:07.425412+0000","last_fullsized":"2024-04-26T20:01:07.425412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422170+0000","last_change":"2024-04-26T20:01:05.928552+0000","last_active":"2024-04-26T20:01:07.422170+0000","last_peered":"2024-04-26T20:01:07.422170+0000","last_clean":"2024-04-26T20:01:07.422170+0000","last_became_active":"2024-04-26T20:01:05.928021+0000","last_became_peered":"2024-04-26T20:01:05.928021+0000","last_unstale":"2024-04-26T20:01:07.422170+0000","last_undegraded":"2024-04-26T20:01:07.422170+0000","last_fullsized":"2024-04-26T20:01:07.422170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934368+0000","last_change":"2024-04-26T20:01:05.936603+0000","last_active":"2024-04-26T20:01:06.934368+0000","last_peered":"2024-04-26T20:01:06.934368+0000","last_clean":"2024-04-26T20:01:06.934368+0000","last_became_active":"2024-04-26T20:01:05.936316+0000","last_became_peered":"2024-04-26T20:01:05.936316+0000","last_unstale":"2024-04-26T20:01:06.934368+0000","last_undegraded":"2024-04-26T20:01:06.934368+0000","last_fullsized":"2024-04-26T20:01:06.934368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423191+0000","last_change":"2024-04-26T20:01:04.900074+0000","last_active":"2024-04-26T20:01:07.423191+0000","last_peered":"2024-04-26T20:01:07.423191+0000","last_clean":"2024-04-26T20:01:07.423191+0000","last_became_active":"2024-04-26T20:01:04.898252+0000","last_became_peered":"2024-04-26T20:01:04.898252+0000","last_unstale":"2024-04-26T20:01:07.423191+0000","last_undegraded":"2024-04-26T20:01:07.423191+0000","last_fullsized":"2024-04-26T20:01:07.423191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422945+0000","last_change":"2024-04-26T20:01:05.922935+0000","last_active":"2024-04-26T20:01:07.422945+0000","last_peered":"2024-04-26T20:01:07.422945+0000","last_clean":"2024-04-26T20:01:07.422945+0000","last_became_active":"2024-04-26T20:01:05.922482+0000","last_became_peered":"2024-04-26T20:01:05.922482+0000","last_unstale":"2024-04-26T20:01:07.422945+0000","last_undegraded":"2024-04-26T20:01:07.422945+0000","last_fullsized":"2024-04-26T20:01:07.422945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934296+0000","last_change":"2024-04-26T20:01:04.902720+0000","last_active":"2024-04-26T20:01:06.934296+0000","last_peered":"2024-04-26T20:01:06.934296+0000","last_clean":"2024-04-26T20:01:06.934296+0000","last_became_active":"2024-04-26T20:01:04.902158+0000","last_became_peered":"2024-04-26T20:01:04.902158+0000","last_unstale":"2024-04-26T20:01:06.934296+0000","last_undegraded":"2024-04-26T20:01:06.934296+0000","last_fullsized":"2024-04-26T20:01:06.934296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928986+0000","last_change":"2024-04-26T20:01:05.926959+0000","last_active":"2024-04-26T20:01:06.928986+0000","last_peered":"2024-04-26T20:01:06.928986+0000","last_clean":"2024-04-26T20:01:06.928986+0000","last_became_active":"2024-04-26T20:01:05.926614+0000","last_became_peered":"2024-04-26T20:01:05.926614+0000","last_unstale":"2024-04-26T20:01:06.928986+0000","last_undegraded":"2024-04-26T20:01:06.928986+0000","last_fullsized":"2024-04-26T20:01:06.928986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420679+0000","last_change":"2024-04-26T20:01:04.897236+0000","last_active":"2024-04-26T20:01:07.420679+0000","last_peered":"2024-04-26T20:01:07.420679+0000","last_clean":"2024-04-26T20:01:07.420679+0000","last_became_active":"2024-04-26T20:01:04.896622+0000","last_became_peered":"2024-04-26T20:01:04.896622+0000","last_unstale":"2024-04-26T20:01:07.420679+0000","last_undegraded":"2024-04-26T20:01:07.420679+0000","last_fullsized":"2024-04-26T20:01:07.420679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:43.508853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932354+0000","last_change":"2024-04-26T20:01:05.934290+0000","last_active":"2024-04-26T20:01:06.932354+0000","last_peered":"2024-04-26T20:01:06.932354+0000","last_clean":"2024-04-26T20:01:06.932354+0000","last_became_active":"2024-04-26T20:01:05.933188+0000","last_became_peered":"2024-04-26T20:01:05.933188+0000","last_unstale":"2024-04-26T20:01:06.932354+0000","last_undegraded":"2024-04-26T20:01:06.932354+0000","last_fullsized":"2024-04-26T20:01:06.932354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929631+0000","last_change":"2024-04-26T20:01:04.879892+0000","last_active":"2024-04-26T20:01:06.929631+0000","last_peered":"2024-04-26T20:01:06.929631+0000","last_clean":"2024-04-26T20:01:06.929631+0000","last_became_active":"2024-04-26T20:01:04.879459+0000","last_became_peered":"2024-04-26T20:01:04.879459+0000","last_unstale":"2024-04-26T20:01:06.929631+0000","last_undegraded":"2024-04-26T20:01:06.929631+0000","last_fullsized":"2024-04-26T20:01:06.929631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423962+0000","last_change":"2024-04-26T20:01:05.930771+0000","last_active":"2024-04-26T20:01:07.423962+0000","last_peered":"2024-04-26T20:01:07.423962+0000","last_clean":"2024-04-26T20:01:07.423962+0000","last_became_active":"2024-04-26T20:01:05.930210+0000","last_became_peered":"2024-04-26T20:01:05.930210+0000","last_unstale":"2024-04-26T20:01:07.423962+0000","last_undegraded":"2024-04-26T20:01:07.423962+0000","last_fullsized":"2024-04-26T20:01:07.423962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929116+0000","last_change":"2024-04-26T20:01:04.899358+0000","last_active":"2024-04-26T20:01:06.929116+0000","last_peered":"2024-04-26T20:01:06.929116+0000","last_clean":"2024-04-26T20:01:06.929116+0000","last_became_active":"2024-04-26T20:01:04.899066+0000","last_became_peered":"2024-04-26T20:01:04.899066+0000","last_unstale":"2024-04-26T20:01:06.929116+0000","last_undegraded":"2024-04-26T20:01:06.929116+0000","last_fullsized":"2024-04-26T20:01:06.929116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421435+0000","last_change":"2024-04-26T20:01:05.925719+0000","last_active":"2024-04-26T20:01:07.421435+0000","last_peered":"2024-04-26T20:01:07.421435+0000","last_clean":"2024-04-26T20:01:07.421435+0000","last_became_active":"2024-04-26T20:01:05.925374+0000","last_became_peered":"2024-04-26T20:01:05.925374+0000","last_unstale":"2024-04-26T20:01:07.421435+0000","last_undegraded":"2024-04-26T20:01:07.421435+0000","last_fullsized":"2024-04-26T20:01:07.421435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929176+0000","last_change":"2024-04-26T20:01:04.896700+0000","last_active":"2024-04-26T20:01:06.929176+0000","last_peered":"2024-04-26T20:01:06.929176+0000","last_clean":"2024-04-26T20:01:06.929176+0000","last_became_active":"2024-04-26T20:01:04.896384+0000","last_became_peered":"2024-04-26T20:01:04.896384+0000","last_unstale":"2024-04-26T20:01:06.929176+0000","last_undegraded":"2024-04-26T20:01:06.929176+0000","last_fullsized":"2024-04-26T20:01:06.929176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:07.281520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928528+0000","last_change":"2024-04-26T20:01:05.928751+0000","last_active":"2024-04-26T20:01:06.928528+0000","last_peered":"2024-04-26T20:01:06.928528+0000","last_clean":"2024-04-26T20:01:06.928528+0000","last_became_active":"2024-04-26T20:01:05.928443+0000","last_became_peered":"2024-04-26T20:01:05.928443+0000","last_unstale":"2024-04-26T20:01:06.928528+0000","last_undegraded":"2024-04-26T20:01:06.928528+0000","last_fullsized":"2024-04-26T20:01:06.928528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420440+0000","last_change":"2024-04-26T20:01:04.891168+0000","last_active":"2024-04-26T20:01:07.420440+0000","last_peered":"2024-04-26T20:01:07.420440+0000","last_clean":"2024-04-26T20:01:07.420440+0000","last_became_active":"2024-04-26T20:01:04.890897+0000","last_became_peered":"2024-04-26T20:01:04.890897+0000","last_unstale":"2024-04-26T20:01:07.420440+0000","last_undegraded":"2024-04-26T20:01:07.420440+0000","last_fullsized":"2024-04-26T20:01:07.420440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:38.348659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928313+0000","last_change":"2024-04-26T20:01:05.918385+0000","last_active":"2024-04-26T20:01:06.928313+0000","last_peered":"2024-04-26T20:01:06.928313+0000","last_clean":"2024-04-26T20:01:06.928313+0000","last_became_active":"2024-04-26T20:01:05.918117+0000","last_became_peered":"2024-04-26T20:01:05.918117+0000","last_unstale":"2024-04-26T20:01:06.928313+0000","last_undegraded":"2024-04-26T20:01:06.928313+0000","last_fullsized":"2024-04-26T20:01:06.928313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928076+0000","last_change":"2024-04-26T20:01:04.900028+0000","last_active":"2024-04-26T20:01:06.928076+0000","last_peered":"2024-04-26T20:01:06.928076+0000","last_clean":"2024-04-26T20:01:06.928076+0000","last_became_active":"2024-04-26T20:01:04.899740+0000","last_became_peered":"2024-04-26T20:01:04.899740+0000","last_unstale":"2024-04-26T20:01:06.928076+0000","last_undegraded":"2024-04-26T20:01:06.928076+0000","last_fullsized":"2024-04-26T20:01:06.928076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424997+0000","last_change":"2024-04-26T20:01:05.924827+0000","last_active":"2024-04-26T20:01:07.424997+0000","last_peered":"2024-04-26T20:01:07.424997+0000","last_clean":"2024-04-26T20:01:07.424997+0000","last_became_active":"2024-04-26T20:01:05.924343+0000","last_became_peered":"2024-04-26T20:01:05.924343+0000","last_unstale":"2024-04-26T20:01:07.424997+0000","last_undegraded":"2024-04-26T20:01:07.424997+0000","last_fullsized":"2024-04-26T20:01:07.424997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421656+0000","last_change":"2024-04-26T20:01:04.904429+0000","last_active":"2024-04-26T20:01:07.421656+0000","last_peered":"2024-04-26T20:01:07.421656+0000","last_clean":"2024-04-26T20:01:07.421656+0000","last_became_active":"2024-04-26T20:01:04.904127+0000","last_became_peered":"2024-04-26T20:01:04.904127+0000","last_unstale":"2024-04-26T20:01:07.421656+0000","last_undegraded":"2024-04-26T20:01:07.421656+0000","last_fullsized":"2024-04-26T20:01:07.421656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:41.449334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421187+0000","last_change":"2024-04-26T20:01:05.930979+0000","last_active":"2024-04-26T20:01:07.421187+0000","last_peered":"2024-04-26T20:01:07.421187+0000","last_clean":"2024-04-26T20:01:07.421187+0000","last_became_active":"2024-04-26T20:01:05.930695+0000","last_became_peered":"2024-04-26T20:01:05.930695+0000","last_unstale":"2024-04-26T20:01:07.421187+0000","last_undegraded":"2024-04-26T20:01:07.421187+0000","last_fullsized":"2024-04-26T20:01:07.421187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:53.308900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934435+0000","last_change":"2024-04-26T20:01:04.902692+0000","last_active":"2024-04-26T20:01:06.934435+0000","last_peered":"2024-04-26T20:01:06.934435+0000","last_clean":"2024-04-26T20:01:06.934435+0000","last_became_active":"2024-04-26T20:01:04.901817+0000","last_became_peered":"2024-04-26T20:01:04.901817+0000","last_unstale":"2024-04-26T20:01:06.934435+0000","last_undegraded":"2024-04-26T20:01:06.934435+0000","last_fullsized":"2024-04-26T20:01:06.934435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421065+0000","last_change":"2024-04-26T20:01:05.922655+0000","last_active":"2024-04-26T20:01:07.421065+0000","last_peered":"2024-04-26T20:01:07.421065+0000","last_clean":"2024-04-26T20:01:07.421065+0000","last_became_active":"2024-04-26T20:01:05.919065+0000","last_became_peered":"2024-04-26T20:01:05.919065+0000","last_unstale":"2024-04-26T20:01:07.421065+0000","last_undegraded":"2024-04-26T20:01:07.421065+0000","last_fullsized":"2024-04-26T20:01:07.421065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423049+0000","last_change":"2024-04-26T20:01:04.903315+0000","last_active":"2024-04-26T20:01:07.423049+0000","last_peered":"2024-04-26T20:01:07.423049+0000","last_clean":"2024-04-26T20:01:07.423049+0000","last_became_active":"2024-04-26T20:01:04.902750+0000","last_became_peered":"2024-04-26T20:01:04.902750+0000","last_unstale":"2024-04-26T20:01:07.423049+0000","last_undegraded":"2024-04-26T20:01:07.423049+0000","last_fullsized":"2024-04-26T20:01:07.423049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422336+0000","last_change":"2024-04-26T20:01:05.926523+0000","last_active":"2024-04-26T20:01:07.422336+0000","last_peered":"2024-04-26T20:01:07.422336+0000","last_clean":"2024-04-26T20:01:07.422336+0000","last_became_active":"2024-04-26T20:01:05.924748+0000","last_became_peered":"2024-04-26T20:01:05.924748+0000","last_unstale":"2024-04-26T20:01:07.422336+0000","last_undegraded":"2024-04-26T20:01:07.422336+0000","last_fullsized":"2024-04-26T20:01:07.422336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421419+0000","last_change":"2024-04-26T20:01:04.896296+0000","last_active":"2024-04-26T20:01:07.421419+0000","last_peered":"2024-04-26T20:01:07.421419+0000","last_clean":"2024-04-26T20:01:07.421419+0000","last_became_active":"2024-04-26T20:01:04.896034+0000","last_became_peered":"2024-04-26T20:01:04.896034+0000","last_unstale":"2024-04-26T20:01:07.421419+0000","last_undegraded":"2024-04-26T20:01:07.421419+0000","last_fullsized":"2024-04-26T20:01:07.421419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.259984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929084+0000","last_change":"2024-04-26T20:01:05.924273+0000","last_active":"2024-04-26T20:01:06.929084+0000","last_peered":"2024-04-26T20:01:06.929084+0000","last_clean":"2024-04-26T20:01:06.929084+0000","last_became_active":"2024-04-26T20:01:05.924002+0000","last_became_peered":"2024-04-26T20:01:05.924002+0000","last_unstale":"2024-04-26T20:01:06.929084+0000","last_undegraded":"2024-04-26T20:01:06.929084+0000","last_fullsized":"2024-04-26T20:01:06.929084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929280+0000","last_change":"2024-04-26T20:01:04.902304+0000","last_active":"2024-04-26T20:01:06.929280+0000","last_peered":"2024-04-26T20:01:06.929280+0000","last_clean":"2024-04-26T20:01:06.929280+0000","last_became_active":"2024-04-26T20:01:04.901986+0000","last_became_peered":"2024-04-26T20:01:04.901986+0000","last_unstale":"2024-04-26T20:01:06.929280+0000","last_undegraded":"2024-04-26T20:01:06.929280+0000","last_fullsized":"2024-04-26T20:01:06.929280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:19.030356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421032+0000","last_change":"2024-04-26T20:01:05.925014+0000","last_active":"2024-04-26T20:01:07.421032+0000","last_peered":"2024-04-26T20:01:07.421032+0000","last_clean":"2024-04-26T20:01:07.421032+0000","last_became_active":"2024-04-26T20:01:05.924734+0000","last_became_peered":"2024-04-26T20:01:05.924734+0000","last_unstale":"2024-04-26T20:01:07.421032+0000","last_undegraded":"2024-04-26T20:01:07.421032+0000","last_fullsized":"2024-04-26T20:01:07.421032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929907+0000","last_change":"2024-04-26T20:01:04.898231+0000","last_active":"2024-04-26T20:01:06.929907+0000","last_peered":"2024-04-26T20:01:06.929907+0000","last_clean":"2024-04-26T20:01:06.929907+0000","last_became_active":"2024-04-26T20:01:04.896868+0000","last_became_peered":"2024-04-26T20:01:04.896868+0000","last_unstale":"2024-04-26T20:01:06.929907+0000","last_undegraded":"2024-04-26T20:01:06.929907+0000","last_fullsized":"2024-04-26T20:01:06.929907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422559+0000","last_change":"2024-04-26T20:01:05.928317+0000","last_active":"2024-04-26T20:01:07.422559+0000","last_peered":"2024-04-26T20:01:07.422559+0000","last_clean":"2024-04-26T20:01:07.422559+0000","last_became_active":"2024-04-26T20:01:05.928030+0000","last_became_peered":"2024-04-26T20:01:05.928030+0000","last_unstale":"2024-04-26T20:01:07.422559+0000","last_undegraded":"2024-04-26T20:01:07.422559+0000","last_fullsized":"2024-04-26T20:01:07.422559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420224+0000","last_change":"2024-04-26T20:01:04.904256+0000","last_active":"2024-04-26T20:01:07.420224+0000","last_peered":"2024-04-26T20:01:07.420224+0000","last_clean":"2024-04-26T20:01:07.420224+0000","last_became_active":"2024-04-26T20:01:04.903998+0000","last_became_peered":"2024-04-26T20:01:04.903998+0000","last_unstale":"2024-04-26T20:01:07.420224+0000","last_undegraded":"2024-04-26T20:01:07.420224+0000","last_fullsized":"2024-04-26T20:01:07.420224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425586+0000","last_change":"2024-04-26T20:01:05.919282+0000","last_active":"2024-04-26T20:01:07.425586+0000","last_peered":"2024-04-26T20:01:07.425586+0000","last_clean":"2024-04-26T20:01:07.425586+0000","last_became_active":"2024-04-26T20:01:05.918943+0000","last_became_peered":"2024-04-26T20:01:05.918943+0000","last_unstale":"2024-04-26T20:01:07.425586+0000","last_undegraded":"2024-04-26T20:01:07.425586+0000","last_fullsized":"2024-04-26T20:01:07.425586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421590+0000","last_change":"2024-04-26T20:01:04.894197+0000","last_active":"2024-04-26T20:01:07.421590+0000","last_peered":"2024-04-26T20:01:07.421590+0000","last_clean":"2024-04-26T20:01:07.421590+0000","last_became_active":"2024-04-26T20:01:04.893919+0000","last_became_peered":"2024-04-26T20:01:04.893919+0000","last_unstale":"2024-04-26T20:01:07.421590+0000","last_undegraded":"2024-04-26T20:01:07.421590+0000","last_fullsized":"2024-04-26T20:01:07.421590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:31.300172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928720+0000","last_change":"2024-04-26T20:01:05.922973+0000","last_active":"2024-04-26T20:01:06.928720+0000","last_peered":"2024-04-26T20:01:06.928720+0000","last_clean":"2024-04-26T20:01:06.928720+0000","last_became_active":"2024-04-26T20:01:05.922657+0000","last_became_peered":"2024-04-26T20:01:05.922657+0000","last_unstale":"2024-04-26T20:01:06.928720+0000","last_undegraded":"2024-04-26T20:01:06.928720+0000","last_fullsized":"2024-04-26T20:01:06.928720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932134+0000","last_change":"2024-04-26T20:01:04.898895+0000","last_active":"2024-04-26T20:01:06.932134+0000","last_peered":"2024-04-26T20:01:06.932134+0000","last_clean":"2024-04-26T20:01:06.932134+0000","last_became_active":"2024-04-26T20:01:04.898535+0000","last_became_peered":"2024-04-26T20:01:04.898535+0000","last_unstale":"2024-04-26T20:01:06.932134+0000","last_undegraded":"2024-04-26T20:01:06.932134+0000","last_fullsized":"2024-04-26T20:01:06.932134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422713+0000","last_change":"2024-04-26T20:01:05.927162+0000","last_active":"2024-04-26T20:01:07.422713+0000","last_peered":"2024-04-26T20:01:07.422713+0000","last_clean":"2024-04-26T20:01:07.422713+0000","last_became_active":"2024-04-26T20:01:05.926785+0000","last_became_peered":"2024-04-26T20:01:05.926785+0000","last_unstale":"2024-04-26T20:01:07.422713+0000","last_undegraded":"2024-04-26T20:01:07.422713+0000","last_fullsized":"2024-04-26T20:01:07.422713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929242+0000","last_change":"2024-04-26T20:01:04.902861+0000","last_active":"2024-04-26T20:01:06.929242+0000","last_peered":"2024-04-26T20:01:06.929242+0000","last_clean":"2024-04-26T20:01:06.929242+0000","last_became_active":"2024-04-26T20:01:04.902393+0000","last_became_peered":"2024-04-26T20:01:04.902393+0000","last_unstale":"2024-04-26T20:01:06.929242+0000","last_undegraded":"2024-04-26T20:01:06.929242+0000","last_fullsized":"2024-04-26T20:01:06.929242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:26.049252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931248+0000","last_change":"2024-04-26T20:01:05.930171+0000","last_active":"2024-04-26T20:01:06.931248+0000","last_peered":"2024-04-26T20:01:06.931248+0000","last_clean":"2024-04-26T20:01:06.931248+0000","last_became_active":"2024-04-26T20:01:05.929737+0000","last_became_peered":"2024-04-26T20:01:05.929737+0000","last_unstale":"2024-04-26T20:01:06.931248+0000","last_undegraded":"2024-04-26T20:01:06.931248+0000","last_fullsized":"2024-04-26T20:01:06.931248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929103+0000","last_change":"2024-04-26T20:01:04.887577+0000","last_active":"2024-04-26T20:01:06.929103+0000","last_peered":"2024-04-26T20:01:06.929103+0000","last_clean":"2024-04-26T20:01:06.929103+0000","last_became_active":"2024-04-26T20:01:04.887059+0000","last_became_peered":"2024-04-26T20:01:04.887059+0000","last_unstale":"2024-04-26T20:01:06.929103+0000","last_undegraded":"2024-04-26T20:01:06.929103+0000","last_fullsized":"2024-04-26T20:01:06.929103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929299+0000","last_change":"2024-04-26T20:01:05.927774+0000","last_active":"2024-04-26T20:01:06.929299+0000","last_peered":"2024-04-26T20:01:06.929299+0000","last_clean":"2024-04-26T20:01:06.929299+0000","last_became_active":"2024-04-26T20:01:05.927459+0000","last_became_peered":"2024-04-26T20:01:05.927459+0000","last_unstale":"2024-04-26T20:01:06.929299+0000","last_undegraded":"2024-04-26T20:01:06.929299+0000","last_fullsized":"2024-04-26T20:01:06.929299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:07.867885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928642+0000","last_change":"2024-04-26T20:01:04.897364+0000","last_active":"2024-04-26T20:01:06.928642+0000","last_peered":"2024-04-26T20:01:06.928642+0000","last_clean":"2024-04-26T20:01:06.928642+0000","last_became_active":"2024-04-26T20:01:04.896186+0000","last_became_peered":"2024-04-26T20:01:04.896186+0000","last_unstale":"2024-04-26T20:01:06.928642+0000","last_undegraded":"2024-04-26T20:01:06.928642+0000","last_fullsized":"2024-04-26T20:01:06.928642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928890+0000","last_change":"2024-04-26T20:01:05.927806+0000","last_active":"2024-04-26T20:01:06.928890+0000","last_peered":"2024-04-26T20:01:06.928890+0000","last_clean":"2024-04-26T20:01:06.928890+0000","last_became_active":"2024-04-26T20:01:05.927383+0000","last_became_peered":"2024-04-26T20:01:05.927383+0000","last_unstale":"2024-04-26T20:01:06.928890+0000","last_undegraded":"2024-04-26T20:01:06.928890+0000","last_fullsized":"2024-04-26T20:01:06.928890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:26.965346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.933989+0000","last_change":"2024-04-26T20:01:04.889881+0000","last_active":"2024-04-26T20:01:06.933989+0000","last_peered":"2024-04-26T20:01:06.933989+0000","last_clean":"2024-04-26T20:01:06.933989+0000","last_became_active":"2024-04-26T20:01:04.889623+0000","last_became_peered":"2024-04-26T20:01:04.889623+0000","last_unstale":"2024-04-26T20:01:06.933989+0000","last_undegraded":"2024-04-26T20:01:06.933989+0000","last_fullsized":"2024-04-26T20:01:06.933989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929270+0000","last_change":"2024-04-26T20:01:05.935015+0000","last_active":"2024-04-26T20:01:06.929270+0000","last_peered":"2024-04-26T20:01:06.929270+0000","last_clean":"2024-04-26T20:01:06.929270+0000","last_became_active":"2024-04-26T20:01:05.934164+0000","last_became_peered":"2024-04-26T20:01:05.934164+0000","last_unstale":"2024-04-26T20:01:06.929270+0000","last_undegraded":"2024-04-26T20:01:06.929270+0000","last_fullsized":"2024-04-26T20:01:06.929270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:10.565596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422516+0000","last_change":"2024-04-26T20:01:04.901055+0000","last_active":"2024-04-26T20:01:07.422516+0000","last_peered":"2024-04-26T20:01:07.422516+0000","last_clean":"2024-04-26T20:01:07.422516+0000","last_became_active":"2024-04-26T20:01:04.900083+0000","last_became_peered":"2024-04-26T20:01:04.900083+0000","last_unstale":"2024-04-26T20:01:07.422516+0000","last_undegraded":"2024-04-26T20:01:07.422516+0000","last_fullsized":"2024-04-26T20:01:07.422516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928234+0000","last_change":"2024-04-26T20:01:05.938285+0000","last_active":"2024-04-26T20:01:06.928234+0000","last_peered":"2024-04-26T20:01:06.928234+0000","last_clean":"2024-04-26T20:01:06.928234+0000","last_became_active":"2024-04-26T20:01:05.934864+0000","last_became_peered":"2024-04-26T20:01:05.934864+0000","last_unstale":"2024-04-26T20:01:06.928234+0000","last_undegraded":"2024-04-26T20:01:06.928234+0000","last_fullsized":"2024-04-26T20:01:06.928234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:08.293626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928147+0000","last_change":"2024-04-26T20:01:04.888976+0000","last_active":"2024-04-26T20:01:06.928147+0000","last_peered":"2024-04-26T20:01:06.928147+0000","last_clean":"2024-04-26T20:01:06.928147+0000","last_became_active":"2024-04-26T20:01:04.888367+0000","last_became_peered":"2024-04-26T20:01:04.888367+0000","last_unstale":"2024-04-26T20:01:06.928147+0000","last_undegraded":"2024-04-26T20:01:06.928147+0000","last_fullsized":"2024-04-26T20:01:06.928147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:51.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421875+0000","last_change":"2024-04-26T20:01:05.922469+0000","last_active":"2024-04-26T20:01:07.421875+0000","last_peered":"2024-04-26T20:01:07.421875+0000","last_clean":"2024-04-26T20:01:07.421875+0000","last_became_active":"2024-04-26T20:01:05.922185+0000","last_became_peered":"2024-04-26T20:01:05.922185+0000","last_unstale":"2024-04-26T20:01:07.421875+0000","last_undegraded":"2024-04-26T20:01:07.421875+0000","last_fullsized":"2024-04-26T20:01:07.421875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420758+0000","last_change":"2024-04-26T20:01:04.906119+0000","last_active":"2024-04-26T20:01:07.420758+0000","last_peered":"2024-04-26T20:01:07.420758+0000","last_clean":"2024-04-26T20:01:07.420758+0000","last_became_active":"2024-04-26T20:01:04.905793+0000","last_became_peered":"2024-04-26T20:01:04.905793+0000","last_unstale":"2024-04-26T20:01:07.420758+0000","last_undegraded":"2024-04-26T20:01:07.420758+0000","last_fullsized":"2024-04-26T20:01:07.420758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928970+0000","last_change":"2024-04-26T20:01:05.928082+0000","last_active":"2024-04-26T20:01:06.928970+0000","last_peered":"2024-04-26T20:01:06.928970+0000","last_clean":"2024-04-26T20:01:06.928970+0000","last_became_active":"2024-04-26T20:01:05.927589+0000","last_became_peered":"2024-04-26T20:01:05.927589+0000","last_unstale":"2024-04-26T20:01:06.928970+0000","last_undegraded":"2024-04-26T20:01:06.928970+0000","last_fullsized":"2024-04-26T20:01:06.928970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:17.326357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932202+0000","last_change":"2024-04-26T20:01:04.897905+0000","last_active":"2024-04-26T20:01:06.932202+0000","last_peered":"2024-04-26T20:01:06.932202+0000","last_clean":"2024-04-26T20:01:06.932202+0000","last_became_active":"2024-04-26T20:01:04.897556+0000","last_became_peered":"2024-04-26T20:01:04.897556+0000","last_unstale":"2024-04-26T20:01:06.932202+0000","last_undegraded":"2024-04-26T20:01:06.932202+0000","last_fullsized":"2024-04-26T20:01:06.932202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424490+0000","last_change":"2024-04-26T20:01:05.926340+0000","last_active":"2024-04-26T20:01:07.424490+0000","last_peered":"2024-04-26T20:01:07.424490+0000","last_clean":"2024-04-26T20:01:07.424490+0000","last_became_active":"2024-04-26T20:01:05.925986+0000","last_became_peered":"2024-04-26T20:01:05.925986+0000","last_unstale":"2024-04-26T20:01:07.424490+0000","last_undegraded":"2024-04-26T20:01:07.424490+0000","last_fullsized":"2024-04-26T20:01:07.424490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934396+0000","last_change":"2024-04-26T20:01:04.888031+0000","last_active":"2024-04-26T20:01:06.934396+0000","last_peered":"2024-04-26T20:01:06.934396+0000","last_clean":"2024-04-26T20:01:06.934396+0000","last_became_active":"2024-04-26T20:01:04.887522+0000","last_became_peered":"2024-04-26T20:01:04.887522+0000","last_unstale":"2024-04-26T20:01:06.934396+0000","last_undegraded":"2024-04-26T20:01:06.934396+0000","last_fullsized":"2024-04-26T20:01:06.934396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934248+0000","last_change":"2024-04-26T20:01:05.939913+0000","last_active":"2024-04-26T20:01:06.934248+0000","last_peered":"2024-04-26T20:01:06.934248+0000","last_clean":"2024-04-26T20:01:06.934248+0000","last_became_active":"2024-04-26T20:01:05.939643+0000","last_became_peered":"2024-04-26T20:01:05.939643+0000","last_unstale":"2024-04-26T20:01:06.934248+0000","last_undegraded":"2024-04-26T20:01:06.934248+0000","last_fullsized":"2024-04-26T20:01:06.934248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931294+0000","last_change":"2024-04-26T20:01:04.882527+0000","last_active":"2024-04-26T20:01:06.931294+0000","last_peered":"2024-04-26T20:01:06.931294+0000","last_clean":"2024-04-26T20:01:06.931294+0000","last_became_active":"2024-04-26T20:01:04.881905+0000","last_became_peered":"2024-04-26T20:01:04.881905+0000","last_unstale":"2024-04-26T20:01:06.931294+0000","last_undegraded":"2024-04-26T20:01:06.931294+0000","last_fullsized":"2024-04-26T20:01:06.931294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422247+0000","last_change":"2024-04-26T20:01:05.926284+0000","last_active":"2024-04-26T20:01:07.422247+0000","last_peered":"2024-04-26T20:01:07.422247+0000","last_clean":"2024-04-26T20:01:07.422247+0000","last_became_active":"2024-04-26T20:01:05.926018+0000","last_became_peered":"2024-04-26T20:01:05.926018+0000","last_unstale":"2024-04-26T20:01:07.422247+0000","last_undegraded":"2024-04-26T20:01:07.422247+0000","last_fullsized":"2024-04-26T20:01:07.422247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:36.109712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934159+0000","last_change":"2024-04-26T20:01:04.898199+0000","last_active":"2024-04-26T20:01:06.934159+0000","last_peered":"2024-04-26T20:01:06.934159+0000","last_clean":"2024-04-26T20:01:06.934159+0000","last_became_active":"2024-04-26T20:01:04.897910+0000","last_became_peered":"2024-04-26T20:01:04.897910+0000","last_unstale":"2024-04-26T20:01:06.934159+0000","last_undegraded":"2024-04-26T20:01:06.934159+0000","last_fullsized":"2024-04-26T20:01:06.934159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934026+0000","last_change":"2024-04-26T20:01:05.931276+0000","last_active":"2024-04-26T20:01:06.934026+0000","last_peered":"2024-04-26T20:01:06.934026+0000","last_clean":"2024-04-26T20:01:06.934026+0000","last_became_active":"2024-04-26T20:01:05.930920+0000","last_became_peered":"2024-04-26T20:01:05.930920+0000","last_unstale":"2024-04-26T20:01:06.934026+0000","last_undegraded":"2024-04-26T20:01:06.934026+0000","last_fullsized":"2024-04-26T20:01:06.934026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930926+0000","last_change":"2024-04-26T20:01:04.898465+0000","last_active":"2024-04-26T20:01:06.930926+0000","last_peered":"2024-04-26T20:01:06.930926+0000","last_clean":"2024-04-26T20:01:06.930926+0000","last_became_active":"2024-04-26T20:01:04.897963+0000","last_became_peered":"2024-04-26T20:01:04.897963+0000","last_unstale":"2024-04-26T20:01:06.930926+0000","last_undegraded":"2024-04-26T20:01:06.930926+0000","last_fullsized":"2024-04-26T20:01:06.930926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420366+0000","last_change":"2024-04-26T20:01:05.914965+0000","last_active":"2024-04-26T20:01:07.420366+0000","last_peered":"2024-04-26T20:01:07.420366+0000","last_clean":"2024-04-26T20:01:07.420366+0000","last_became_active":"2024-04-26T20:01:05.914644+0000","last_became_peered":"2024-04-26T20:01:05.914644+0000","last_unstale":"2024-04-26T20:01:07.420366+0000","last_undegraded":"2024-04-26T20:01:07.420366+0000","last_fullsized":"2024-04-26T20:01:07.420366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928398+0000","last_change":"2024-04-26T20:01:04.894822+0000","last_active":"2024-04-26T20:01:06.928398+0000","last_peered":"2024-04-26T20:01:06.928398+0000","last_clean":"2024-04-26T20:01:06.928398+0000","last_became_active":"2024-04-26T20:01:04.893421+0000","last_became_peered":"2024-04-26T20:01:04.893421+0000","last_unstale":"2024-04-26T20:01:06.928398+0000","last_undegraded":"2024-04-26T20:01:06.928398+0000","last_fullsized":"2024-04-26T20:01:06.928398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:54.615275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928104+0000","last_change":"2024-04-26T20:01:05.924736+0000","last_active":"2024-04-26T20:01:06.928104+0000","last_peered":"2024-04-26T20:01:06.928104+0000","last_clean":"2024-04-26T20:01:06.928104+0000","last_became_active":"2024-04-26T20:01:05.922449+0000","last_became_peered":"2024-04-26T20:01:05.922449+0000","last_unstale":"2024-04-26T20:01:06.928104+0000","last_undegraded":"2024-04-26T20:01:06.928104+0000","last_fullsized":"2024-04-26T20:01:06.928104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420394+0000","last_change":"2024-04-26T20:01:04.904249+0000","last_active":"2024-04-26T20:01:07.420394+0000","last_peered":"2024-04-26T20:01:07.420394+0000","last_clean":"2024-04-26T20:01:07.420394+0000","last_became_active":"2024-04-26T20:01:04.903995+0000","last_became_peered":"2024-04-26T20:01:04.903995+0000","last_unstale":"2024-04-26T20:01:07.420394+0000","last_undegraded":"2024-04-26T20:01:07.420394+0000","last_fullsized":"2024-04-26T20:01:07.420394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424191+0000","last_change":"2024-04-26T20:01:05.938661+0000","last_active":"2024-04-26T20:01:07.424191+0000","last_peered":"2024-04-26T20:01:07.424191+0000","last_clean":"2024-04-26T20:01:07.424191+0000","last_became_active":"2024-04-26T20:01:05.938340+0000","last_became_peered":"2024-04-26T20:01:05.938340+0000","last_unstale":"2024-04-26T20:01:07.424191+0000","last_undegraded":"2024-04-26T20:01:07.424191+0000","last_fullsized":"2024-04-26T20:01:07.424191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421989+0000","last_change":"2024-04-26T20:01:04.903742+0000","last_active":"2024-04-26T20:01:07.421989+0000","last_peered":"2024-04-26T20:01:07.421989+0000","last_clean":"2024-04-26T20:01:07.421989+0000","last_became_active":"2024-04-26T20:01:04.903299+0000","last_became_peered":"2024-04-26T20:01:04.903299+0000","last_unstale":"2024-04-26T20:01:07.421989+0000","last_undegraded":"2024-04-26T20:01:07.421989+0000","last_fullsized":"2024-04-26T20:01:07.421989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423373+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:07.423373+0000","last_peered":"2024-04-26T20:01:07.423373+0000","last_clean":"2024-04-26T20:01:07.423373+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:07.423373+0000","last_undegraded":"2024-04-26T20:01:07.423373+0000","last_fullsized":"2024-04-26T20:01:07.423373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":737138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:01:15.330 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:01:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.540+0000 7fa043200640 1 Processor -- start 2024-04-26T20:01:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.540+0000 7fa043200640 1 -- start start 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa043200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa0440d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa043200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa044070680 0x7fa044071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa043200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa044070c40 con 0x7fa04415d000 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa043200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa044071850 con 0x7fa044070680 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa043200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa044070f20 con 0x7fa0440d32d0 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa041e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa04415d000 0x7fa0440d2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34368/0 (socket says 172.21.15.88:34368) 2024-04-26T20:01:15.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.542+0000 7fa041e00640 1 -- 172.21.15.88:0/4258021999 learned_addr learned my addr 172.21.15.88:0/4258021999 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:15.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa042800640 1 --2- 172.21.15.88:0/4258021999 >> v2:172.21.15.132:3300/0 conn(0x7fa044070680 0x7fa044071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa03be00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa0440d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504843035 0 0) 0x7fa044070c40 con 0x7fa04415d000 2024-04-26T20:01:15.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa028003520 con 0x7fa04415d000 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3435473891 0 0) 0x7fa028003520 con 0x7fa04415d000 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 msgr2=0x7fa0440d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa0440d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 >> v2:172.21.15.132:3300/0 conn(0x7fa044070680 msgr2=0x7fa044071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 --2- 172.21.15.88:0/4258021999 >> v2:172.21.15.132:3300/0 conn(0x7fa044070680 0x7fa044071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa044071200 con 0x7fa04415d000 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa034001670 con 0x7fa04415d000 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 >> v1:172.21.15.88:6789/0 conn(0x7fa04415d000 legacy=0x7fa0440d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0280039b0 0x7fa028003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa028004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa028004d40 0x7fa0280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa044071850 con 0x7fa0280039b0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa044070f20 con 0x7fa028004d40 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.543+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa02800b9f0 con 0x7fa0440d32d0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0280039b0 0x7fa028003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa042800640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa028004d40 0x7fa0280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa03be00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa028004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 -- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 msgr2=0x7fa028004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0440d32d0 0x7fa028004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 -- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa028004d40 msgr2=0x7fa0280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa028004d40 0x7fa0280051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.544+0000 7fa041e00640 1 -- 172.21.15.88:0/4258021999 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa028003520 con 0x7fa0280039b0 2024-04-26T20:01:15.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.545+0000 7fa041e00640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0280039b0 0x7fa028003dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa03400c160 tx=0x7fa028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54faf1f8fa5dbabe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:15.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.545+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034007c80 con 0x7fa0280039b0 2024-04-26T20:01:15.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.545+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa034005af0 con 0x7fa0280039b0 2024-04-26T20:01:15.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.545+0000 7fa041400640 1 -- 172.21.15.88:0/4258021999 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034007070 con 0x7fa0280039b0 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 -- 172.21.15.88:0/4258021999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0280039b0 msgr2=0x7fa028003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 --2- 172.21.15.88:0/4258021999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0280039b0 0x7fa028003dd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa03400c160 tx=0x7fa028005830 comp rx=0 tx=0).stop 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa042800640 1 -- 172.21.15.88:0/4258021999 reap_dead start 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 -- 172.21.15.88:0/4258021999 shutdown_connections 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 -- 172.21.15.88:0/4258021999 >> 172.21.15.88:0/4258021999 conn(0x7fa04409ba50 msgr2=0x7fa04409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 -- 172.21.15.88:0/4258021999 shutdown_connections 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.546+0000 7fa043200640 1 -- 172.21.15.88:0/4258021999 wait complete. 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.548+0000 7fa043200640 1 Processor -- start 2024-04-26T20:01:15.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.548+0000 7fa043200640 1 -- start start 2024-04-26T20:01:15.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0280039b0 0x7fa0441f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440c65e0 0x7fa0440c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440d01e0 con 0x7fa0440b69d0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa0440d0060 con 0x7fa0440c65e0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.549+0000 7fa043200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa0440d0360 con 0x7fa0280039b0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58236/0 (socket says 172.21.15.88:58236) 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 -- 172.21.15.88:0/3007917180 learned_addr learned my addr 172.21.15.88:0/3007917180 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa041e00640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0280039b0 0x7fa0441f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa03be00640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440c65e0 0x7fa0440c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 -- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0280039b0 msgr2=0x7fa0441f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0280039b0 0x7fa0441f2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 -- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440c65e0 msgr2=0x7fa0440c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440c65e0 0x7fa0440c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 -- 172.21.15.88:0/3007917180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440c9490 con 0x7fa0440b69d0 2024-04-26T20:01:15.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.550+0000 7fa042800640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:15.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.551+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c01c070 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.551+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03c017400 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.551+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0177f0 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.551+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441f26b0 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.551+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440b5c10 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.552+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440693c0 con 0x7fa0440b69d0 2024-04-26T20:01:15.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.553+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa03c005d10 con 0x7fa0440b69d0 2024-04-26T20:01:15.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.554+0000 7fa03a000640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:15.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.554+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa03c08bcf0 con 0x7fa0440b69d0 2024-04-26T20:01:15.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.554+0000 7fa041e00640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:15.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.555+0000 7fa041e00640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa034032000 tx=0x7fa034007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:15.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.558+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa03c012070 con 0x7fa0440b69d0 2024-04-26T20:01:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.752+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa04409c4c0 con 0x7fa02006c8f0 2024-04-26T20:01:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.760+0000 7fa03a000640 1 -- 172.21.15.88:0/3007917180 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251955 (secure 0 0 0) 0x7fa04409c4c0 con 0x7fa02006c8f0 2024-04-26T20:01:15.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:15.766 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:01:15.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 msgr2=0x7fa02006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 0x7fa02006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa034032000 tx=0x7fa034007000 comp rx=0 tx=0).stop 2024-04-26T20:01:15.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 msgr2=0x7fa0440b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:15.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c00f4a0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 shutdown_connections 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0440b69d0 0x7fa0440b6e50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0440c65e0 0x7fa0440c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa02006c8f0 0x7fa02006edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 --2- 172.21.15.88:0/3007917180 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa0280039b0 0x7fa0441f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.766+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 >> 172.21.15.88:0/3007917180 conn(0x7fa04409ba50 msgr2=0x7fa04415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.767+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 shutdown_connections 2024-04-26T20:01:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:15.767+0000 7fa043200640 1 -- 172.21.15.88:0/3007917180 wait complete. 2024-04-26T20:01:15.785 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T20:01:14.966612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220340,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754380,"statfs":{"total":773094113280,"available":772868485120,"internally_reserved":0,"allocated":4235264,"data_stored":2359716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.074966"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422513+0000","last_change":"2024-04-26T20:01:05.919081+0000","last_active":"2024-04-26T20:01:07.422513+0000","last_peered":"2024-04-26T20:01:07.422513+0000","last_clean":"2024-04-26T20:01:07.422513+0000","last_became_active":"2024-04-26T20:01:05.916236+0000","last_became_peered":"2024-04-26T20:01:05.916236+0000","last_unstale":"2024-04-26T20:01:07.422513+0000","last_undegraded":"2024-04-26T20:01:07.422513+0000","last_fullsized":"2024-04-26T20:01:07.422513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.876764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421185+0000","last_change":"2024-04-26T20:01:05.924948+0000","last_active":"2024-04-26T20:01:07.421185+0000","last_peered":"2024-04-26T20:01:07.421185+0000","last_clean":"2024-04-26T20:01:07.421185+0000","last_became_active":"2024-04-26T20:01:05.924322+0000","last_became_peered":"2024-04-26T20:01:05.924322+0000","last_unstale":"2024-04-26T20:01:07.421185+0000","last_undegraded":"2024-04-26T20:01:07.421185+0000","last_fullsized":"2024-04-26T20:01:07.421185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:39.001253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422789+0000","last_change":"2024-04-26T20:01:05.925708+0000","last_active":"2024-04-26T20:01:07.422789+0000","last_peered":"2024-04-26T20:01:07.422789+0000","last_clean":"2024-04-26T20:01:07.422789+0000","last_became_active":"2024-04-26T20:01:05.924331+0000","last_became_peered":"2024-04-26T20:01:05.924331+0000","last_unstale":"2024-04-26T20:01:07.422789+0000","last_undegraded":"2024-04-26T20:01:07.422789+0000","last_fullsized":"2024-04-26T20:01:07.422789+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930927+0000","last_change":"2024-04-26T20:01:05.930430+0000","last_active":"2024-04-26T20:01:06.930927+0000","last_peered":"2024-04-26T20:01:06.930927+0000","last_clean":"2024-04-26T20:01:06.930927+0000","last_became_active":"2024-04-26T20:01:05.930139+0000","last_became_peered":"2024-04-26T20:01:05.930139+0000","last_unstale":"2024-04-26T20:01:06.930927+0000","last_undegraded":"2024-04-26T20:01:06.930927+0000","last_fullsized":"2024-04-26T20:01:06.930927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928615+0000","last_change":"2024-04-26T20:01:05.934500+0000","last_active":"2024-04-26T20:01:06.928615+0000","last_peered":"2024-04-26T20:01:06.928615+0000","last_clean":"2024-04-26T20:01:06.928615+0000","last_became_active":"2024-04-26T20:01:05.933310+0000","last_became_peered":"2024-04-26T20:01:05.933310+0000","last_unstale":"2024-04-26T20:01:06.928615+0000","last_undegraded":"2024-04-26T20:01:06.928615+0000","last_fullsized":"2024-04-26T20:01:06.928615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.323134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420434+0000","last_change":"2024-04-26T20:01:05.912926+0000","last_active":"2024-04-26T20:01:07.420434+0000","last_peered":"2024-04-26T20:01:07.420434+0000","last_clean":"2024-04-26T20:01:07.420434+0000","last_became_active":"2024-04-26T20:01:05.912473+0000","last_became_peered":"2024-04-26T20:01:05.912473+0000","last_unstale":"2024-04-26T20:01:07.420434+0000","last_undegraded":"2024-04-26T20:01:07.420434+0000","last_fullsized":"2024-04-26T20:01:07.420434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:51.470568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423817+0000","last_change":"2024-04-26T20:01:05.928943+0000","last_active":"2024-04-26T20:01:07.423817+0000","last_peered":"2024-04-26T20:01:07.423817+0000","last_clean":"2024-04-26T20:01:07.423817+0000","last_became_active":"2024-04-26T20:01:05.928434+0000","last_became_peered":"2024-04-26T20:01:05.928434+0000","last_unstale":"2024-04-26T20:01:07.423817+0000","last_undegraded":"2024-04-26T20:01:07.423817+0000","last_fullsized":"2024-04-26T20:01:07.423817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421731+0000","last_change":"2024-04-26T20:01:05.940280+0000","last_active":"2024-04-26T20:01:07.421731+0000","last_peered":"2024-04-26T20:01:07.421731+0000","last_clean":"2024-04-26T20:01:07.421731+0000","last_became_active":"2024-04-26T20:01:05.940034+0000","last_became_peered":"2024-04-26T20:01:05.940034+0000","last_unstale":"2024-04-26T20:01:07.421731+0000","last_undegraded":"2024-04-26T20:01:07.421731+0000","last_fullsized":"2024-04-26T20:01:07.421731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929681+0000","last_change":"2024-04-26T20:01:05.945356+0000","last_active":"2024-04-26T20:01:06.929681+0000","last_peered":"2024-04-26T20:01:06.929681+0000","last_clean":"2024-04-26T20:01:06.929681+0000","last_became_active":"2024-04-26T20:01:05.945132+0000","last_became_peered":"2024-04-26T20:01:05.945132+0000","last_unstale":"2024-04-26T20:01:06.929681+0000","last_undegraded":"2024-04-26T20:01:06.929681+0000","last_fullsized":"2024-04-26T20:01:06.929681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931906+0000","last_change":"2024-04-26T20:01:05.926545+0000","last_active":"2024-04-26T20:01:06.931906+0000","last_peered":"2024-04-26T20:01:06.931906+0000","last_clean":"2024-04-26T20:01:06.931906+0000","last_became_active":"2024-04-26T20:01:05.924721+0000","last_became_peered":"2024-04-26T20:01:05.924721+0000","last_unstale":"2024-04-26T20:01:06.931906+0000","last_undegraded":"2024-04-26T20:01:06.931906+0000","last_fullsized":"2024-04-26T20:01:06.931906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424805+0000","last_change":"2024-04-26T20:01:05.939937+0000","last_active":"2024-04-26T20:01:07.424805+0000","last_peered":"2024-04-26T20:01:07.424805+0000","last_clean":"2024-04-26T20:01:07.424805+0000","last_became_active":"2024-04-26T20:01:05.939644+0000","last_became_peered":"2024-04-26T20:01:05.939644+0000","last_unstale":"2024-04-26T20:01:07.424805+0000","last_undegraded":"2024-04-26T20:01:07.424805+0000","last_fullsized":"2024-04-26T20:01:07.424805+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423650+0000","last_change":"2024-04-26T20:01:05.919891+0000","last_active":"2024-04-26T20:01:07.423650+0000","last_peered":"2024-04-26T20:01:07.423650+0000","last_clean":"2024-04-26T20:01:07.423650+0000","last_became_active":"2024-04-26T20:01:05.919631+0000","last_became_peered":"2024-04-26T20:01:05.919631+0000","last_unstale":"2024-04-26T20:01:07.423650+0000","last_undegraded":"2024-04-26T20:01:07.423650+0000","last_fullsized":"2024-04-26T20:01:07.423650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928767+0000","last_change":"2024-04-26T20:01:05.927158+0000","last_active":"2024-04-26T20:01:06.928767+0000","last_peered":"2024-04-26T20:01:06.928767+0000","last_clean":"2024-04-26T20:01:06.928767+0000","last_became_active":"2024-04-26T20:01:05.926783+0000","last_became_peered":"2024-04-26T20:01:05.926783+0000","last_unstale":"2024-04-26T20:01:06.928767+0000","last_undegraded":"2024-04-26T20:01:06.928767+0000","last_fullsized":"2024-04-26T20:01:06.928767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:01.310367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420598+0000","last_change":"2024-04-26T20:01:05.940579+0000","last_active":"2024-04-26T20:01:07.420598+0000","last_peered":"2024-04-26T20:01:07.420598+0000","last_clean":"2024-04-26T20:01:07.420598+0000","last_became_active":"2024-04-26T20:01:05.940304+0000","last_became_peered":"2024-04-26T20:01:05.940304+0000","last_unstale":"2024-04-26T20:01:07.420598+0000","last_undegraded":"2024-04-26T20:01:07.420598+0000","last_fullsized":"2024-04-26T20:01:07.420598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:21.271559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421597+0000","last_change":"2024-04-26T20:01:05.923735+0000","last_active":"2024-04-26T20:01:07.421597+0000","last_peered":"2024-04-26T20:01:07.421597+0000","last_clean":"2024-04-26T20:01:07.421597+0000","last_became_active":"2024-04-26T20:01:05.923381+0000","last_became_peered":"2024-04-26T20:01:05.923381+0000","last_unstale":"2024-04-26T20:01:07.421597+0000","last_undegraded":"2024-04-26T20:01:07.421597+0000","last_fullsized":"2024-04-26T20:01:07.421597+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.469882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423136+0000","last_change":"2024-04-26T20:01:05.923995+0000","last_active":"2024-04-26T20:01:07.423136+0000","last_peered":"2024-04-26T20:01:07.423136+0000","last_clean":"2024-04-26T20:01:07.423136+0000","last_became_active":"2024-04-26T20:01:05.923638+0000","last_became_peered":"2024-04-26T20:01:05.923638+0000","last_unstale":"2024-04-26T20:01:07.423136+0000","last_undegraded":"2024-04-26T20:01:07.423136+0000","last_fullsized":"2024-04-26T20:01:07.423136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424734+0000","last_change":"2024-04-26T20:01:05.927247+0000","last_active":"2024-04-26T20:01:07.424734+0000","last_peered":"2024-04-26T20:01:07.424734+0000","last_clean":"2024-04-26T20:01:07.424734+0000","last_became_active":"2024-04-26T20:01:05.924925+0000","last_became_peered":"2024-04-26T20:01:05.924925+0000","last_unstale":"2024-04-26T20:01:07.424734+0000","last_undegraded":"2024-04-26T20:01:07.424734+0000","last_fullsized":"2024-04-26T20:01:07.424734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421254+0000","last_change":"2024-04-26T20:01:05.926554+0000","last_active":"2024-04-26T20:01:07.421254+0000","last_peered":"2024-04-26T20:01:07.421254+0000","last_clean":"2024-04-26T20:01:07.421254+0000","last_became_active":"2024-04-26T20:01:05.926287+0000","last_became_peered":"2024-04-26T20:01:05.926287+0000","last_unstale":"2024-04-26T20:01:07.421254+0000","last_undegraded":"2024-04-26T20:01:07.421254+0000","last_fullsized":"2024-04-26T20:01:07.421254+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:45:43.379844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421059+0000","last_change":"2024-04-26T20:01:05.914996+0000","last_active":"2024-04-26T20:01:07.421059+0000","last_peered":"2024-04-26T20:01:07.421059+0000","last_clean":"2024-04-26T20:01:07.421059+0000","last_became_active":"2024-04-26T20:01:05.914692+0000","last_became_peered":"2024-04-26T20:01:05.914692+0000","last_unstale":"2024-04-26T20:01:07.421059+0000","last_undegraded":"2024-04-26T20:01:07.421059+0000","last_fullsized":"2024-04-26T20:01:07.421059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929118+0000","last_change":"2024-04-26T20:01:05.943085+0000","last_active":"2024-04-26T20:01:06.929118+0000","last_peered":"2024-04-26T20:01:06.929118+0000","last_clean":"2024-04-26T20:01:06.929118+0000","last_became_active":"2024-04-26T20:01:05.940081+0000","last_became_peered":"2024-04-26T20:01:05.940081+0000","last_unstale":"2024-04-26T20:01:06.929118+0000","last_undegraded":"2024-04-26T20:01:06.929118+0000","last_fullsized":"2024-04-26T20:01:06.929118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:22.045959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928666+0000","last_change":"2024-04-26T20:01:05.932062+0000","last_active":"2024-04-26T20:01:06.928666+0000","last_peered":"2024-04-26T20:01:06.928666+0000","last_clean":"2024-04-26T20:01:06.928666+0000","last_became_active":"2024-04-26T20:01:05.931734+0000","last_became_peered":"2024-04-26T20:01:05.931734+0000","last_unstale":"2024-04-26T20:01:06.928666+0000","last_undegraded":"2024-04-26T20:01:06.928666+0000","last_fullsized":"2024-04-26T20:01:06.928666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928891+0000","last_change":"2024-04-26T20:01:05.926555+0000","last_active":"2024-04-26T20:01:06.928891+0000","last_peered":"2024-04-26T20:01:06.928891+0000","last_clean":"2024-04-26T20:01:06.928891+0000","last_became_active":"2024-04-26T20:01:05.925941+0000","last_became_peered":"2024-04-26T20:01:05.925941+0000","last_unstale":"2024-04-26T20:01:06.928891+0000","last_undegraded":"2024-04-26T20:01:06.928891+0000","last_fullsized":"2024-04-26T20:01:06.928891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:44.520959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934038+0000","last_change":"2024-04-26T20:01:05.935796+0000","last_active":"2024-04-26T20:01:06.934038+0000","last_peered":"2024-04-26T20:01:06.934038+0000","last_clean":"2024-04-26T20:01:06.934038+0000","last_became_active":"2024-04-26T20:01:05.935441+0000","last_became_peered":"2024-04-26T20:01:05.935441+0000","last_unstale":"2024-04-26T20:01:06.934038+0000","last_undegraded":"2024-04-26T20:01:06.934038+0000","last_fullsized":"2024-04-26T20:01:06.934038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420885+0000","last_change":"2024-04-26T20:01:05.937693+0000","last_active":"2024-04-26T20:01:07.420885+0000","last_peered":"2024-04-26T20:01:07.420885+0000","last_clean":"2024-04-26T20:01:07.420885+0000","last_became_active":"2024-04-26T20:01:05.937313+0000","last_became_peered":"2024-04-26T20:01:05.937313+0000","last_unstale":"2024-04-26T20:01:07.420885+0000","last_undegraded":"2024-04-26T20:01:07.420885+0000","last_fullsized":"2024-04-26T20:01:07.420885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932008+0000","last_change":"2024-04-26T20:01:05.932177+0000","last_active":"2024-04-26T20:01:06.932008+0000","last_peered":"2024-04-26T20:01:06.932008+0000","last_clean":"2024-04-26T20:01:06.932008+0000","last_became_active":"2024-04-26T20:01:05.931858+0000","last_became_peered":"2024-04-26T20:01:05.931858+0000","last_unstale":"2024-04-26T20:01:06.932008+0000","last_undegraded":"2024-04-26T20:01:06.932008+0000","last_fullsized":"2024-04-26T20:01:06.932008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421612+0000","last_change":"2024-04-26T20:01:05.925756+0000","last_active":"2024-04-26T20:01:07.421612+0000","last_peered":"2024-04-26T20:01:07.421612+0000","last_clean":"2024-04-26T20:01:07.421612+0000","last_became_active":"2024-04-26T20:01:05.925478+0000","last_became_peered":"2024-04-26T20:01:05.925478+0000","last_unstale":"2024-04-26T20:01:07.421612+0000","last_undegraded":"2024-04-26T20:01:07.421612+0000","last_fullsized":"2024-04-26T20:01:07.421612+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:59.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932258+0000","last_change":"2024-04-26T20:01:05.934504+0000","last_active":"2024-04-26T20:01:06.932258+0000","last_peered":"2024-04-26T20:01:06.932258+0000","last_clean":"2024-04-26T20:01:06.932258+0000","last_became_active":"2024-04-26T20:01:05.934031+0000","last_became_peered":"2024-04-26T20:01:05.934031+0000","last_unstale":"2024-04-26T20:01:06.932258+0000","last_undegraded":"2024-04-26T20:01:06.932258+0000","last_fullsized":"2024-04-26T20:01:06.932258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:40:21.201837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424485+0000","last_change":"2024-04-26T20:01:05.931477+0000","last_active":"2024-04-26T20:01:07.424485+0000","last_peered":"2024-04-26T20:01:07.424485+0000","last_clean":"2024-04-26T20:01:07.424485+0000","last_became_active":"2024-04-26T20:01:05.930780+0000","last_became_peered":"2024-04-26T20:01:05.930780+0000","last_unstale":"2024-04-26T20:01:07.424485+0000","last_undegraded":"2024-04-26T20:01:07.424485+0000","last_fullsized":"2024-04-26T20:01:07.424485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928987+0000","last_change":"2024-04-26T20:01:05.921417+0000","last_active":"2024-04-26T20:01:06.928987+0000","last_peered":"2024-04-26T20:01:06.928987+0000","last_clean":"2024-04-26T20:01:06.928987+0000","last_became_active":"2024-04-26T20:01:05.920831+0000","last_became_peered":"2024-04-26T20:01:05.920831+0000","last_unstale":"2024-04-26T20:01:06.928987+0000","last_undegraded":"2024-04-26T20:01:06.928987+0000","last_fullsized":"2024-04-26T20:01:06.928987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:42.461439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424420+0000","last_change":"2024-04-26T20:01:05.930517+0000","last_active":"2024-04-26T20:01:07.424420+0000","last_peered":"2024-04-26T20:01:07.424420+0000","last_clean":"2024-04-26T20:01:07.424420+0000","last_became_active":"2024-04-26T20:01:05.929798+0000","last_became_peered":"2024-04-26T20:01:05.929798+0000","last_unstale":"2024-04-26T20:01:07.424420+0000","last_undegraded":"2024-04-26T20:01:07.424420+0000","last_fullsized":"2024-04-26T20:01:07.424420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425412+0000","last_change":"2024-04-26T20:01:05.929755+0000","last_active":"2024-04-26T20:01:07.425412+0000","last_peered":"2024-04-26T20:01:07.425412+0000","last_clean":"2024-04-26T20:01:07.425412+0000","last_became_active":"2024-04-26T20:01:05.929417+0000","last_became_peered":"2024-04-26T20:01:05.929417+0000","last_unstale":"2024-04-26T20:01:07.425412+0000","last_undegraded":"2024-04-26T20:01:07.425412+0000","last_fullsized":"2024-04-26T20:01:07.425412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422170+0000","last_change":"2024-04-26T20:01:05.928552+0000","last_active":"2024-04-26T20:01:07.422170+0000","last_peered":"2024-04-26T20:01:07.422170+0000","last_clean":"2024-04-26T20:01:07.422170+0000","last_became_active":"2024-04-26T20:01:05.928021+0000","last_became_peered":"2024-04-26T20:01:05.928021+0000","last_unstale":"2024-04-26T20:01:07.422170+0000","last_undegraded":"2024-04-26T20:01:07.422170+0000","last_fullsized":"2024-04-26T20:01:07.422170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:52.429668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934368+0000","last_change":"2024-04-26T20:01:05.936603+0000","last_active":"2024-04-26T20:01:06.934368+0000","last_peered":"2024-04-26T20:01:06.934368+0000","last_clean":"2024-04-26T20:01:06.934368+0000","last_became_active":"2024-04-26T20:01:05.936316+0000","last_became_peered":"2024-04-26T20:01:05.936316+0000","last_unstale":"2024-04-26T20:01:06.934368+0000","last_undegraded":"2024-04-26T20:01:06.934368+0000","last_fullsized":"2024-04-26T20:01:06.934368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423191+0000","last_change":"2024-04-26T20:01:04.900074+0000","last_active":"2024-04-26T20:01:07.423191+0000","last_peered":"2024-04-26T20:01:07.423191+0000","last_clean":"2024-04-26T20:01:07.423191+0000","last_became_active":"2024-04-26T20:01:04.898252+0000","last_became_peered":"2024-04-26T20:01:04.898252+0000","last_unstale":"2024-04-26T20:01:07.423191+0000","last_undegraded":"2024-04-26T20:01:07.423191+0000","last_fullsized":"2024-04-26T20:01:07.423191+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422945+0000","last_change":"2024-04-26T20:01:05.922935+0000","last_active":"2024-04-26T20:01:07.422945+0000","last_peered":"2024-04-26T20:01:07.422945+0000","last_clean":"2024-04-26T20:01:07.422945+0000","last_became_active":"2024-04-26T20:01:05.922482+0000","last_became_peered":"2024-04-26T20:01:05.922482+0000","last_unstale":"2024-04-26T20:01:07.422945+0000","last_undegraded":"2024-04-26T20:01:07.422945+0000","last_fullsized":"2024-04-26T20:01:07.422945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934296+0000","last_change":"2024-04-26T20:01:04.902720+0000","last_active":"2024-04-26T20:01:06.934296+0000","last_peered":"2024-04-26T20:01:06.934296+0000","last_clean":"2024-04-26T20:01:06.934296+0000","last_became_active":"2024-04-26T20:01:04.902158+0000","last_became_peered":"2024-04-26T20:01:04.902158+0000","last_unstale":"2024-04-26T20:01:06.934296+0000","last_undegraded":"2024-04-26T20:01:06.934296+0000","last_fullsized":"2024-04-26T20:01:06.934296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928986+0000","last_change":"2024-04-26T20:01:05.926959+0000","last_active":"2024-04-26T20:01:06.928986+0000","last_peered":"2024-04-26T20:01:06.928986+0000","last_clean":"2024-04-26T20:01:06.928986+0000","last_became_active":"2024-04-26T20:01:05.926614+0000","last_became_peered":"2024-04-26T20:01:05.926614+0000","last_unstale":"2024-04-26T20:01:06.928986+0000","last_undegraded":"2024-04-26T20:01:06.928986+0000","last_fullsized":"2024-04-26T20:01:06.928986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:56:59.291169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420679+0000","last_change":"2024-04-26T20:01:04.897236+0000","last_active":"2024-04-26T20:01:07.420679+0000","last_peered":"2024-04-26T20:01:07.420679+0000","last_clean":"2024-04-26T20:01:07.420679+0000","last_became_active":"2024-04-26T20:01:04.896622+0000","last_became_peered":"2024-04-26T20:01:04.896622+0000","last_unstale":"2024-04-26T20:01:07.420679+0000","last_undegraded":"2024-04-26T20:01:07.420679+0000","last_fullsized":"2024-04-26T20:01:07.420679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:43.508853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932354+0000","last_change":"2024-04-26T20:01:05.934290+0000","last_active":"2024-04-26T20:01:06.932354+0000","last_peered":"2024-04-26T20:01:06.932354+0000","last_clean":"2024-04-26T20:01:06.932354+0000","last_became_active":"2024-04-26T20:01:05.933188+0000","last_became_peered":"2024-04-26T20:01:05.933188+0000","last_unstale":"2024-04-26T20:01:06.932354+0000","last_undegraded":"2024-04-26T20:01:06.932354+0000","last_fullsized":"2024-04-26T20:01:06.932354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929631+0000","last_change":"2024-04-26T20:01:04.879892+0000","last_active":"2024-04-26T20:01:06.929631+0000","last_peered":"2024-04-26T20:01:06.929631+0000","last_clean":"2024-04-26T20:01:06.929631+0000","last_became_active":"2024-04-26T20:01:04.879459+0000","last_became_peered":"2024-04-26T20:01:04.879459+0000","last_unstale":"2024-04-26T20:01:06.929631+0000","last_undegraded":"2024-04-26T20:01:06.929631+0000","last_fullsized":"2024-04-26T20:01:06.929631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423962+0000","last_change":"2024-04-26T20:01:05.930771+0000","last_active":"2024-04-26T20:01:07.423962+0000","last_peered":"2024-04-26T20:01:07.423962+0000","last_clean":"2024-04-26T20:01:07.423962+0000","last_became_active":"2024-04-26T20:01:05.930210+0000","last_became_peered":"2024-04-26T20:01:05.930210+0000","last_unstale":"2024-04-26T20:01:07.423962+0000","last_undegraded":"2024-04-26T20:01:07.423962+0000","last_fullsized":"2024-04-26T20:01:07.423962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929116+0000","last_change":"2024-04-26T20:01:04.899358+0000","last_active":"2024-04-26T20:01:06.929116+0000","last_peered":"2024-04-26T20:01:06.929116+0000","last_clean":"2024-04-26T20:01:06.929116+0000","last_became_active":"2024-04-26T20:01:04.899066+0000","last_became_peered":"2024-04-26T20:01:04.899066+0000","last_unstale":"2024-04-26T20:01:06.929116+0000","last_undegraded":"2024-04-26T20:01:06.929116+0000","last_fullsized":"2024-04-26T20:01:06.929116+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:56.178279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421435+0000","last_change":"2024-04-26T20:01:05.925719+0000","last_active":"2024-04-26T20:01:07.421435+0000","last_peered":"2024-04-26T20:01:07.421435+0000","last_clean":"2024-04-26T20:01:07.421435+0000","last_became_active":"2024-04-26T20:01:05.925374+0000","last_became_peered":"2024-04-26T20:01:05.925374+0000","last_unstale":"2024-04-26T20:01:07.421435+0000","last_undegraded":"2024-04-26T20:01:07.421435+0000","last_fullsized":"2024-04-26T20:01:07.421435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929176+0000","last_change":"2024-04-26T20:01:04.896700+0000","last_active":"2024-04-26T20:01:06.929176+0000","last_peered":"2024-04-26T20:01:06.929176+0000","last_clean":"2024-04-26T20:01:06.929176+0000","last_became_active":"2024-04-26T20:01:04.896384+0000","last_became_peered":"2024-04-26T20:01:04.896384+0000","last_unstale":"2024-04-26T20:01:06.929176+0000","last_undegraded":"2024-04-26T20:01:06.929176+0000","last_fullsized":"2024-04-26T20:01:06.929176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:07.281520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928528+0000","last_change":"2024-04-26T20:01:05.928751+0000","last_active":"2024-04-26T20:01:06.928528+0000","last_peered":"2024-04-26T20:01:06.928528+0000","last_clean":"2024-04-26T20:01:06.928528+0000","last_became_active":"2024-04-26T20:01:05.928443+0000","last_became_peered":"2024-04-26T20:01:05.928443+0000","last_unstale":"2024-04-26T20:01:06.928528+0000","last_undegraded":"2024-04-26T20:01:06.928528+0000","last_fullsized":"2024-04-26T20:01:06.928528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:32.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420440+0000","last_change":"2024-04-26T20:01:04.891168+0000","last_active":"2024-04-26T20:01:07.420440+0000","last_peered":"2024-04-26T20:01:07.420440+0000","last_clean":"2024-04-26T20:01:07.420440+0000","last_became_active":"2024-04-26T20:01:04.890897+0000","last_became_peered":"2024-04-26T20:01:04.890897+0000","last_unstale":"2024-04-26T20:01:07.420440+0000","last_undegraded":"2024-04-26T20:01:07.420440+0000","last_fullsized":"2024-04-26T20:01:07.420440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:38.348659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928313+0000","last_change":"2024-04-26T20:01:05.918385+0000","last_active":"2024-04-26T20:01:06.928313+0000","last_peered":"2024-04-26T20:01:06.928313+0000","last_clean":"2024-04-26T20:01:06.928313+0000","last_became_active":"2024-04-26T20:01:05.918117+0000","last_became_peered":"2024-04-26T20:01:05.918117+0000","last_unstale":"2024-04-26T20:01:06.928313+0000","last_undegraded":"2024-04-26T20:01:06.928313+0000","last_fullsized":"2024-04-26T20:01:06.928313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928076+0000","last_change":"2024-04-26T20:01:04.900028+0000","last_active":"2024-04-26T20:01:06.928076+0000","last_peered":"2024-04-26T20:01:06.928076+0000","last_clean":"2024-04-26T20:01:06.928076+0000","last_became_active":"2024-04-26T20:01:04.899740+0000","last_became_peered":"2024-04-26T20:01:04.899740+0000","last_unstale":"2024-04-26T20:01:06.928076+0000","last_undegraded":"2024-04-26T20:01:06.928076+0000","last_fullsized":"2024-04-26T20:01:06.928076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:29.311029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424997+0000","last_change":"2024-04-26T20:01:05.924827+0000","last_active":"2024-04-26T20:01:07.424997+0000","last_peered":"2024-04-26T20:01:07.424997+0000","last_clean":"2024-04-26T20:01:07.424997+0000","last_became_active":"2024-04-26T20:01:05.924343+0000","last_became_peered":"2024-04-26T20:01:05.924343+0000","last_unstale":"2024-04-26T20:01:07.424997+0000","last_undegraded":"2024-04-26T20:01:07.424997+0000","last_fullsized":"2024-04-26T20:01:07.424997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:56.855594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421656+0000","last_change":"2024-04-26T20:01:04.904429+0000","last_active":"2024-04-26T20:01:07.421656+0000","last_peered":"2024-04-26T20:01:07.421656+0000","last_clean":"2024-04-26T20:01:07.421656+0000","last_became_active":"2024-04-26T20:01:04.904127+0000","last_became_peered":"2024-04-26T20:01:04.904127+0000","last_unstale":"2024-04-26T20:01:07.421656+0000","last_undegraded":"2024-04-26T20:01:07.421656+0000","last_fullsized":"2024-04-26T20:01:07.421656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:41.449334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421187+0000","last_change":"2024-04-26T20:01:05.930979+0000","last_active":"2024-04-26T20:01:07.421187+0000","last_peered":"2024-04-26T20:01:07.421187+0000","last_clean":"2024-04-26T20:01:07.421187+0000","last_became_active":"2024-04-26T20:01:05.930695+0000","last_became_peered":"2024-04-26T20:01:05.930695+0000","last_unstale":"2024-04-26T20:01:07.421187+0000","last_undegraded":"2024-04-26T20:01:07.421187+0000","last_fullsized":"2024-04-26T20:01:07.421187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:53.308900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934435+0000","last_change":"2024-04-26T20:01:04.902692+0000","last_active":"2024-04-26T20:01:06.934435+0000","last_peered":"2024-04-26T20:01:06.934435+0000","last_clean":"2024-04-26T20:01:06.934435+0000","last_became_active":"2024-04-26T20:01:04.901817+0000","last_became_peered":"2024-04-26T20:01:04.901817+0000","last_unstale":"2024-04-26T20:01:06.934435+0000","last_undegraded":"2024-04-26T20:01:06.934435+0000","last_fullsized":"2024-04-26T20:01:06.934435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421065+0000","last_change":"2024-04-26T20:01:05.922655+0000","last_active":"2024-04-26T20:01:07.421065+0000","last_peered":"2024-04-26T20:01:07.421065+0000","last_clean":"2024-04-26T20:01:07.421065+0000","last_became_active":"2024-04-26T20:01:05.919065+0000","last_became_peered":"2024-04-26T20:01:05.919065+0000","last_unstale":"2024-04-26T20:01:07.421065+0000","last_undegraded":"2024-04-26T20:01:07.421065+0000","last_fullsized":"2024-04-26T20:01:07.421065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:23:13.535180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423049+0000","last_change":"2024-04-26T20:01:04.903315+0000","last_active":"2024-04-26T20:01:07.423049+0000","last_peered":"2024-04-26T20:01:07.423049+0000","last_clean":"2024-04-26T20:01:07.423049+0000","last_became_active":"2024-04-26T20:01:04.902750+0000","last_became_peered":"2024-04-26T20:01:04.902750+0000","last_unstale":"2024-04-26T20:01:07.423049+0000","last_undegraded":"2024-04-26T20:01:07.423049+0000","last_fullsized":"2024-04-26T20:01:07.423049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422336+0000","last_change":"2024-04-26T20:01:05.926523+0000","last_active":"2024-04-26T20:01:07.422336+0000","last_peered":"2024-04-26T20:01:07.422336+0000","last_clean":"2024-04-26T20:01:07.422336+0000","last_became_active":"2024-04-26T20:01:05.924748+0000","last_became_peered":"2024-04-26T20:01:05.924748+0000","last_unstale":"2024-04-26T20:01:07.422336+0000","last_undegraded":"2024-04-26T20:01:07.422336+0000","last_fullsized":"2024-04-26T20:01:07.422336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421419+0000","last_change":"2024-04-26T20:01:04.896296+0000","last_active":"2024-04-26T20:01:07.421419+0000","last_peered":"2024-04-26T20:01:07.421419+0000","last_clean":"2024-04-26T20:01:07.421419+0000","last_became_active":"2024-04-26T20:01:04.896034+0000","last_became_peered":"2024-04-26T20:01:04.896034+0000","last_unstale":"2024-04-26T20:01:07.421419+0000","last_undegraded":"2024-04-26T20:01:07.421419+0000","last_fullsized":"2024-04-26T20:01:07.421419+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:24.259984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929084+0000","last_change":"2024-04-26T20:01:05.924273+0000","last_active":"2024-04-26T20:01:06.929084+0000","last_peered":"2024-04-26T20:01:06.929084+0000","last_clean":"2024-04-26T20:01:06.929084+0000","last_became_active":"2024-04-26T20:01:05.924002+0000","last_became_peered":"2024-04-26T20:01:05.924002+0000","last_unstale":"2024-04-26T20:01:06.929084+0000","last_undegraded":"2024-04-26T20:01:06.929084+0000","last_fullsized":"2024-04-26T20:01:06.929084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:01:51.701604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929280+0000","last_change":"2024-04-26T20:01:04.902304+0000","last_active":"2024-04-26T20:01:06.929280+0000","last_peered":"2024-04-26T20:01:06.929280+0000","last_clean":"2024-04-26T20:01:06.929280+0000","last_became_active":"2024-04-26T20:01:04.901986+0000","last_became_peered":"2024-04-26T20:01:04.901986+0000","last_unstale":"2024-04-26T20:01:06.929280+0000","last_undegraded":"2024-04-26T20:01:06.929280+0000","last_fullsized":"2024-04-26T20:01:06.929280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:19.030356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421032+0000","last_change":"2024-04-26T20:01:05.925014+0000","last_active":"2024-04-26T20:01:07.421032+0000","last_peered":"2024-04-26T20:01:07.421032+0000","last_clean":"2024-04-26T20:01:07.421032+0000","last_became_active":"2024-04-26T20:01:05.924734+0000","last_became_peered":"2024-04-26T20:01:05.924734+0000","last_unstale":"2024-04-26T20:01:07.421032+0000","last_undegraded":"2024-04-26T20:01:07.421032+0000","last_fullsized":"2024-04-26T20:01:07.421032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929907+0000","last_change":"2024-04-26T20:01:04.898231+0000","last_active":"2024-04-26T20:01:06.929907+0000","last_peered":"2024-04-26T20:01:06.929907+0000","last_clean":"2024-04-26T20:01:06.929907+0000","last_became_active":"2024-04-26T20:01:04.896868+0000","last_became_peered":"2024-04-26T20:01:04.896868+0000","last_unstale":"2024-04-26T20:01:06.929907+0000","last_undegraded":"2024-04-26T20:01:06.929907+0000","last_fullsized":"2024-04-26T20:01:06.929907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:06.855780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422559+0000","last_change":"2024-04-26T20:01:05.928317+0000","last_active":"2024-04-26T20:01:07.422559+0000","last_peered":"2024-04-26T20:01:07.422559+0000","last_clean":"2024-04-26T20:01:07.422559+0000","last_became_active":"2024-04-26T20:01:05.928030+0000","last_became_peered":"2024-04-26T20:01:05.928030+0000","last_unstale":"2024-04-26T20:01:07.422559+0000","last_undegraded":"2024-04-26T20:01:07.422559+0000","last_fullsized":"2024-04-26T20:01:07.422559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420224+0000","last_change":"2024-04-26T20:01:04.904256+0000","last_active":"2024-04-26T20:01:07.420224+0000","last_peered":"2024-04-26T20:01:07.420224+0000","last_clean":"2024-04-26T20:01:07.420224+0000","last_became_active":"2024-04-26T20:01:04.903998+0000","last_became_peered":"2024-04-26T20:01:04.903998+0000","last_unstale":"2024-04-26T20:01:07.420224+0000","last_undegraded":"2024-04-26T20:01:07.420224+0000","last_fullsized":"2024-04-26T20:01:07.420224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.425586+0000","last_change":"2024-04-26T20:01:05.919282+0000","last_active":"2024-04-26T20:01:07.425586+0000","last_peered":"2024-04-26T20:01:07.425586+0000","last_clean":"2024-04-26T20:01:07.425586+0000","last_became_active":"2024-04-26T20:01:05.918943+0000","last_became_peered":"2024-04-26T20:01:05.918943+0000","last_unstale":"2024-04-26T20:01:07.425586+0000","last_undegraded":"2024-04-26T20:01:07.425586+0000","last_fullsized":"2024-04-26T20:01:07.425586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:25.272090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421590+0000","last_change":"2024-04-26T20:01:04.894197+0000","last_active":"2024-04-26T20:01:07.421590+0000","last_peered":"2024-04-26T20:01:07.421590+0000","last_clean":"2024-04-26T20:01:07.421590+0000","last_became_active":"2024-04-26T20:01:04.893919+0000","last_became_peered":"2024-04-26T20:01:04.893919+0000","last_unstale":"2024-04-26T20:01:07.421590+0000","last_undegraded":"2024-04-26T20:01:07.421590+0000","last_fullsized":"2024-04-26T20:01:07.421590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:31.300172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928720+0000","last_change":"2024-04-26T20:01:05.922973+0000","last_active":"2024-04-26T20:01:06.928720+0000","last_peered":"2024-04-26T20:01:06.928720+0000","last_clean":"2024-04-26T20:01:06.928720+0000","last_became_active":"2024-04-26T20:01:05.922657+0000","last_became_peered":"2024-04-26T20:01:05.922657+0000","last_unstale":"2024-04-26T20:01:06.928720+0000","last_undegraded":"2024-04-26T20:01:06.928720+0000","last_fullsized":"2024-04-26T20:01:06.928720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932134+0000","last_change":"2024-04-26T20:01:04.898895+0000","last_active":"2024-04-26T20:01:06.932134+0000","last_peered":"2024-04-26T20:01:06.932134+0000","last_clean":"2024-04-26T20:01:06.932134+0000","last_became_active":"2024-04-26T20:01:04.898535+0000","last_became_peered":"2024-04-26T20:01:04.898535+0000","last_unstale":"2024-04-26T20:01:06.932134+0000","last_undegraded":"2024-04-26T20:01:06.932134+0000","last_fullsized":"2024-04-26T20:01:06.932134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422713+0000","last_change":"2024-04-26T20:01:05.927162+0000","last_active":"2024-04-26T20:01:07.422713+0000","last_peered":"2024-04-26T20:01:07.422713+0000","last_clean":"2024-04-26T20:01:07.422713+0000","last_became_active":"2024-04-26T20:01:05.926785+0000","last_became_peered":"2024-04-26T20:01:05.926785+0000","last_unstale":"2024-04-26T20:01:07.422713+0000","last_undegraded":"2024-04-26T20:01:07.422713+0000","last_fullsized":"2024-04-26T20:01:07.422713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:43:27.064656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929242+0000","last_change":"2024-04-26T20:01:04.902861+0000","last_active":"2024-04-26T20:01:06.929242+0000","last_peered":"2024-04-26T20:01:06.929242+0000","last_clean":"2024-04-26T20:01:06.929242+0000","last_became_active":"2024-04-26T20:01:04.902393+0000","last_became_peered":"2024-04-26T20:01:04.902393+0000","last_unstale":"2024-04-26T20:01:06.929242+0000","last_undegraded":"2024-04-26T20:01:06.929242+0000","last_fullsized":"2024-04-26T20:01:06.929242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:26.049252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931248+0000","last_change":"2024-04-26T20:01:05.930171+0000","last_active":"2024-04-26T20:01:06.931248+0000","last_peered":"2024-04-26T20:01:06.931248+0000","last_clean":"2024-04-26T20:01:06.931248+0000","last_became_active":"2024-04-26T20:01:05.929737+0000","last_became_peered":"2024-04-26T20:01:05.929737+0000","last_unstale":"2024-04-26T20:01:06.931248+0000","last_undegraded":"2024-04-26T20:01:06.931248+0000","last_fullsized":"2024-04-26T20:01:06.931248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:58.997521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929103+0000","last_change":"2024-04-26T20:01:04.887577+0000","last_active":"2024-04-26T20:01:06.929103+0000","last_peered":"2024-04-26T20:01:06.929103+0000","last_clean":"2024-04-26T20:01:06.929103+0000","last_became_active":"2024-04-26T20:01:04.887059+0000","last_became_peered":"2024-04-26T20:01:04.887059+0000","last_unstale":"2024-04-26T20:01:06.929103+0000","last_undegraded":"2024-04-26T20:01:06.929103+0000","last_fullsized":"2024-04-26T20:01:06.929103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929299+0000","last_change":"2024-04-26T20:01:05.927774+0000","last_active":"2024-04-26T20:01:06.929299+0000","last_peered":"2024-04-26T20:01:06.929299+0000","last_clean":"2024-04-26T20:01:06.929299+0000","last_became_active":"2024-04-26T20:01:05.927459+0000","last_became_peered":"2024-04-26T20:01:05.927459+0000","last_unstale":"2024-04-26T20:01:06.929299+0000","last_undegraded":"2024-04-26T20:01:06.929299+0000","last_fullsized":"2024-04-26T20:01:06.929299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:07.867885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928642+0000","last_change":"2024-04-26T20:01:04.897364+0000","last_active":"2024-04-26T20:01:06.928642+0000","last_peered":"2024-04-26T20:01:06.928642+0000","last_clean":"2024-04-26T20:01:06.928642+0000","last_became_active":"2024-04-26T20:01:04.896186+0000","last_became_peered":"2024-04-26T20:01:04.896186+0000","last_unstale":"2024-04-26T20:01:06.928642+0000","last_undegraded":"2024-04-26T20:01:06.928642+0000","last_fullsized":"2024-04-26T20:01:06.928642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:25.953241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928890+0000","last_change":"2024-04-26T20:01:05.927806+0000","last_active":"2024-04-26T20:01:06.928890+0000","last_peered":"2024-04-26T20:01:06.928890+0000","last_clean":"2024-04-26T20:01:06.928890+0000","last_became_active":"2024-04-26T20:01:05.927383+0000","last_became_peered":"2024-04-26T20:01:05.927383+0000","last_unstale":"2024-04-26T20:01:06.928890+0000","last_undegraded":"2024-04-26T20:01:06.928890+0000","last_fullsized":"2024-04-26T20:01:06.928890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:26.965346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.933989+0000","last_change":"2024-04-26T20:01:04.889881+0000","last_active":"2024-04-26T20:01:06.933989+0000","last_peered":"2024-04-26T20:01:06.933989+0000","last_clean":"2024-04-26T20:01:06.933989+0000","last_became_active":"2024-04-26T20:01:04.889623+0000","last_became_peered":"2024-04-26T20:01:04.889623+0000","last_unstale":"2024-04-26T20:01:06.933989+0000","last_undegraded":"2024-04-26T20:01:06.933989+0000","last_fullsized":"2024-04-26T20:01:06.933989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.929270+0000","last_change":"2024-04-26T20:01:05.935015+0000","last_active":"2024-04-26T20:01:06.929270+0000","last_peered":"2024-04-26T20:01:06.929270+0000","last_clean":"2024-04-26T20:01:06.929270+0000","last_became_active":"2024-04-26T20:01:05.934164+0000","last_became_peered":"2024-04-26T20:01:05.934164+0000","last_unstale":"2024-04-26T20:01:06.929270+0000","last_undegraded":"2024-04-26T20:01:06.929270+0000","last_fullsized":"2024-04-26T20:01:06.929270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:10.565596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422516+0000","last_change":"2024-04-26T20:01:04.901055+0000","last_active":"2024-04-26T20:01:07.422516+0000","last_peered":"2024-04-26T20:01:07.422516+0000","last_clean":"2024-04-26T20:01:07.422516+0000","last_became_active":"2024-04-26T20:01:04.900083+0000","last_became_peered":"2024-04-26T20:01:04.900083+0000","last_unstale":"2024-04-26T20:01:07.422516+0000","last_undegraded":"2024-04-26T20:01:07.422516+0000","last_fullsized":"2024-04-26T20:01:07.422516+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:16.314251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928234+0000","last_change":"2024-04-26T20:01:05.938285+0000","last_active":"2024-04-26T20:01:06.928234+0000","last_peered":"2024-04-26T20:01:06.928234+0000","last_clean":"2024-04-26T20:01:06.928234+0000","last_became_active":"2024-04-26T20:01:05.934864+0000","last_became_peered":"2024-04-26T20:01:05.934864+0000","last_unstale":"2024-04-26T20:01:06.928234+0000","last_undegraded":"2024-04-26T20:01:06.928234+0000","last_fullsized":"2024-04-26T20:01:06.928234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:08.293626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928147+0000","last_change":"2024-04-26T20:01:04.888976+0000","last_active":"2024-04-26T20:01:06.928147+0000","last_peered":"2024-04-26T20:01:06.928147+0000","last_clean":"2024-04-26T20:01:06.928147+0000","last_became_active":"2024-04-26T20:01:04.888367+0000","last_became_peered":"2024-04-26T20:01:04.888367+0000","last_unstale":"2024-04-26T20:01:06.928147+0000","last_undegraded":"2024-04-26T20:01:06.928147+0000","last_fullsized":"2024-04-26T20:01:06.928147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:51.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421875+0000","last_change":"2024-04-26T20:01:05.922469+0000","last_active":"2024-04-26T20:01:07.421875+0000","last_peered":"2024-04-26T20:01:07.421875+0000","last_clean":"2024-04-26T20:01:07.421875+0000","last_became_active":"2024-04-26T20:01:05.922185+0000","last_became_peered":"2024-04-26T20:01:05.922185+0000","last_unstale":"2024-04-26T20:01:07.421875+0000","last_undegraded":"2024-04-26T20:01:07.421875+0000","last_fullsized":"2024-04-26T20:01:07.421875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420758+0000","last_change":"2024-04-26T20:01:04.906119+0000","last_active":"2024-04-26T20:01:07.420758+0000","last_peered":"2024-04-26T20:01:07.420758+0000","last_clean":"2024-04-26T20:01:07.420758+0000","last_became_active":"2024-04-26T20:01:04.905793+0000","last_became_peered":"2024-04-26T20:01:04.905793+0000","last_unstale":"2024-04-26T20:01:07.420758+0000","last_undegraded":"2024-04-26T20:01:07.420758+0000","last_fullsized":"2024-04-26T20:01:07.420758+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928970+0000","last_change":"2024-04-26T20:01:05.928082+0000","last_active":"2024-04-26T20:01:06.928970+0000","last_peered":"2024-04-26T20:01:06.928970+0000","last_clean":"2024-04-26T20:01:06.928970+0000","last_became_active":"2024-04-26T20:01:05.927589+0000","last_became_peered":"2024-04-26T20:01:05.927589+0000","last_unstale":"2024-04-26T20:01:06.928970+0000","last_undegraded":"2024-04-26T20:01:06.928970+0000","last_fullsized":"2024-04-26T20:01:06.928970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:07:17.326357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.932202+0000","last_change":"2024-04-26T20:01:04.897905+0000","last_active":"2024-04-26T20:01:06.932202+0000","last_peered":"2024-04-26T20:01:06.932202+0000","last_clean":"2024-04-26T20:01:06.932202+0000","last_became_active":"2024-04-26T20:01:04.897556+0000","last_became_peered":"2024-04-26T20:01:04.897556+0000","last_unstale":"2024-04-26T20:01:06.932202+0000","last_undegraded":"2024-04-26T20:01:06.932202+0000","last_fullsized":"2024-04-26T20:01:06.932202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424490+0000","last_change":"2024-04-26T20:01:05.926340+0000","last_active":"2024-04-26T20:01:07.424490+0000","last_peered":"2024-04-26T20:01:07.424490+0000","last_clean":"2024-04-26T20:01:07.424490+0000","last_became_active":"2024-04-26T20:01:05.925986+0000","last_became_peered":"2024-04-26T20:01:05.925986+0000","last_unstale":"2024-04-26T20:01:07.424490+0000","last_undegraded":"2024-04-26T20:01:07.424490+0000","last_fullsized":"2024-04-26T20:01:07.424490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:17.834975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934396+0000","last_change":"2024-04-26T20:01:04.888031+0000","last_active":"2024-04-26T20:01:06.934396+0000","last_peered":"2024-04-26T20:01:06.934396+0000","last_clean":"2024-04-26T20:01:06.934396+0000","last_became_active":"2024-04-26T20:01:04.887522+0000","last_became_peered":"2024-04-26T20:01:04.887522+0000","last_unstale":"2024-04-26T20:01:06.934396+0000","last_undegraded":"2024-04-26T20:01:06.934396+0000","last_fullsized":"2024-04-26T20:01:06.934396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:01.864658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934248+0000","last_change":"2024-04-26T20:01:05.939913+0000","last_active":"2024-04-26T20:01:06.934248+0000","last_peered":"2024-04-26T20:01:06.934248+0000","last_clean":"2024-04-26T20:01:06.934248+0000","last_became_active":"2024-04-26T20:01:05.939643+0000","last_became_peered":"2024-04-26T20:01:05.939643+0000","last_unstale":"2024-04-26T20:01:06.934248+0000","last_undegraded":"2024-04-26T20:01:06.934248+0000","last_fullsized":"2024-04-26T20:01:06.934248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:54:29.473993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.931294+0000","last_change":"2024-04-26T20:01:04.882527+0000","last_active":"2024-04-26T20:01:06.931294+0000","last_peered":"2024-04-26T20:01:06.931294+0000","last_clean":"2024-04-26T20:01:06.931294+0000","last_became_active":"2024-04-26T20:01:04.881905+0000","last_became_peered":"2024-04-26T20:01:04.881905+0000","last_unstale":"2024-04-26T20:01:06.931294+0000","last_undegraded":"2024-04-26T20:01:06.931294+0000","last_fullsized":"2024-04-26T20:01:06.931294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:03.608878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.422247+0000","last_change":"2024-04-26T20:01:05.926284+0000","last_active":"2024-04-26T20:01:07.422247+0000","last_peered":"2024-04-26T20:01:07.422247+0000","last_clean":"2024-04-26T20:01:07.422247+0000","last_became_active":"2024-04-26T20:01:05.926018+0000","last_became_peered":"2024-04-26T20:01:05.926018+0000","last_unstale":"2024-04-26T20:01:07.422247+0000","last_undegraded":"2024-04-26T20:01:07.422247+0000","last_fullsized":"2024-04-26T20:01:07.422247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:36.109712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934159+0000","last_change":"2024-04-26T20:01:04.898199+0000","last_active":"2024-04-26T20:01:06.934159+0000","last_peered":"2024-04-26T20:01:06.934159+0000","last_clean":"2024-04-26T20:01:06.934159+0000","last_became_active":"2024-04-26T20:01:04.897910+0000","last_became_peered":"2024-04-26T20:01:04.897910+0000","last_unstale":"2024-04-26T20:01:06.934159+0000","last_undegraded":"2024-04-26T20:01:06.934159+0000","last_fullsized":"2024-04-26T20:01:06.934159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:56.836909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.934026+0000","last_change":"2024-04-26T20:01:05.931276+0000","last_active":"2024-04-26T20:01:06.934026+0000","last_peered":"2024-04-26T20:01:06.934026+0000","last_clean":"2024-04-26T20:01:06.934026+0000","last_became_active":"2024-04-26T20:01:05.930920+0000","last_became_peered":"2024-04-26T20:01:05.930920+0000","last_unstale":"2024-04-26T20:01:06.934026+0000","last_undegraded":"2024-04-26T20:01:06.934026+0000","last_fullsized":"2024-04-26T20:01:06.934026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:33:23.021565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.930926+0000","last_change":"2024-04-26T20:01:04.898465+0000","last_active":"2024-04-26T20:01:06.930926+0000","last_peered":"2024-04-26T20:01:06.930926+0000","last_clean":"2024-04-26T20:01:06.930926+0000","last_became_active":"2024-04-26T20:01:04.897963+0000","last_became_peered":"2024-04-26T20:01:04.897963+0000","last_unstale":"2024-04-26T20:01:06.930926+0000","last_undegraded":"2024-04-26T20:01:06.930926+0000","last_fullsized":"2024-04-26T20:01:06.930926+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:21.033853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420366+0000","last_change":"2024-04-26T20:01:05.914965+0000","last_active":"2024-04-26T20:01:07.420366+0000","last_peered":"2024-04-26T20:01:07.420366+0000","last_clean":"2024-04-26T20:01:07.420366+0000","last_became_active":"2024-04-26T20:01:05.914644+0000","last_became_peered":"2024-04-26T20:01:05.914644+0000","last_unstale":"2024-04-26T20:01:07.420366+0000","last_undegraded":"2024-04-26T20:01:07.420366+0000","last_fullsized":"2024-04-26T20:01:07.420366+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:07:41.670175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928398+0000","last_change":"2024-04-26T20:01:04.894822+0000","last_active":"2024-04-26T20:01:06.928398+0000","last_peered":"2024-04-26T20:01:06.928398+0000","last_clean":"2024-04-26T20:01:06.928398+0000","last_became_active":"2024-04-26T20:01:04.893421+0000","last_became_peered":"2024-04-26T20:01:04.893421+0000","last_unstale":"2024-04-26T20:01:06.928398+0000","last_undegraded":"2024-04-26T20:01:06.928398+0000","last_fullsized":"2024-04-26T20:01:06.928398+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:54.615275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:06.928104+0000","last_change":"2024-04-26T20:01:05.924736+0000","last_active":"2024-04-26T20:01:06.928104+0000","last_peered":"2024-04-26T20:01:06.928104+0000","last_clean":"2024-04-26T20:01:06.928104+0000","last_became_active":"2024-04-26T20:01:05.922449+0000","last_became_peered":"2024-04-26T20:01:05.922449+0000","last_unstale":"2024-04-26T20:01:06.928104+0000","last_undegraded":"2024-04-26T20:01:06.928104+0000","last_fullsized":"2024-04-26T20:01:06.928104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.360764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.420394+0000","last_change":"2024-04-26T20:01:04.904249+0000","last_active":"2024-04-26T20:01:07.420394+0000","last_peered":"2024-04-26T20:01:07.420394+0000","last_clean":"2024-04-26T20:01:07.420394+0000","last_became_active":"2024-04-26T20:01:04.903995+0000","last_became_peered":"2024-04-26T20:01:04.903995+0000","last_unstale":"2024-04-26T20:01:07.420394+0000","last_undegraded":"2024-04-26T20:01:07.420394+0000","last_fullsized":"2024-04-26T20:01:07.420394+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:35.097607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.424191+0000","last_change":"2024-04-26T20:01:05.938661+0000","last_active":"2024-04-26T20:01:07.424191+0000","last_peered":"2024-04-26T20:01:07.424191+0000","last_clean":"2024-04-26T20:01:07.424191+0000","last_became_active":"2024-04-26T20:01:05.938340+0000","last_became_peered":"2024-04-26T20:01:05.938340+0000","last_unstale":"2024-04-26T20:01:07.424191+0000","last_undegraded":"2024-04-26T20:01:07.424191+0000","last_fullsized":"2024-04-26T20:01:07.424191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:04.857041+0000","last_clean_scrub_stamp":"2024-04-26T20:01:04.857041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:53.786486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.421989+0000","last_change":"2024-04-26T20:01:04.903742+0000","last_active":"2024-04-26T20:01:07.421989+0000","last_peered":"2024-04-26T20:01:07.421989+0000","last_clean":"2024-04-26T20:01:07.421989+0000","last_became_active":"2024-04-26T20:01:04.903299+0000","last_became_peered":"2024-04-26T20:01:04.903299+0000","last_unstale":"2024-04-26T20:01:07.421989+0000","last_undegraded":"2024-04-26T20:01:07.421989+0000","last_fullsized":"2024-04-26T20:01:07.421989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:01:03.844936+0000","last_clean_scrub_stamp":"2024-04-26T20:01:03.844936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:00.298262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T20:01:07.423373+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:07.423373+0000","last_peered":"2024-04-26T20:01:07.423373+0000","last_clean":"2024-04-26T20:01:07.423373+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:07.423373+0000","last_undegraded":"2024-04-26T20:01:07.423373+0000","last_fullsized":"2024-04-26T20:01:07.423373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":737228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":389120,"data_stored":148404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":737138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:01:15.790 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:01:15.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:01:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.038+0000 7f1a7de00640 1 Processor -- start 2024-04-26T20:01:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.038+0000 7f1a7de00640 1 -- start start 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a7de00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1a78073790 0x7f1a7806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a7de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a7815d550 0x7f1a780b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a7de00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7815da90 con 0x7f1a78073b70 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a7de00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7815dd70 con 0x7f1a78073790 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a7de00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7815e050 con 0x7f1a7815d550 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a77400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1a78073b70 0x7f1a78072660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34376/0 (socket says 172.21.15.88:34376) 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a77400640 1 -- 172.21.15.88:0/1589757071 learned_addr learned my addr 172.21.15.88:0/1589757071 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.040+0000 7f1a76a00640 1 --2- 172.21.15.88:0/1589757071 >> v2:172.21.15.88:3300/0 conn(0x7f1a78073790 0x7f1a7806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a76000640 1 --2- 172.21.15.88:0/1589757071 >> v2:172.21.15.132:3300/0 conn(0x7f1a7815d550 0x7f1a780b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a76a00640 1 -- 172.21.15.88:0/1589757071 >> v2:172.21.15.132:3300/0 conn(0x7f1a7815d550 msgr2=0x7f1a780b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a76a00640 1 --2- 172.21.15.88:0/1589757071 >> v2:172.21.15.132:3300/0 conn(0x7f1a7815d550 0x7f1a780b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a75600640 1 -- 172.21.15.88:0/1589757071 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138429129 0 0) 0x7f1a7815da90 con 0x7f1a78073b70 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a76a00640 1 -- 172.21.15.88:0/1589757071 >> v1:172.21.15.88:6789/0 conn(0x7f1a78073b70 legacy=0x7f1a78072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.041+0000 7f1a76a00640 1 -- 172.21.15.88:0/1589757071 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7815e330 con 0x7f1a78073790 2024-04-26T20:01:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.042+0000 7f1a76a00640 1 --2- 172.21.15.88:0/1589757071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a7806dde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1a780d1d70 tx=0x7f1a6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7603f76a25186d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a75600640 1 -- 172.21.15.88:0/1589757071 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68037070 con 0x7f1a78073790 2024-04-26T20:01:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a75600640 1 -- 172.21.15.88:0/1589757071 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68032440 con 0x7f1a78073790 2024-04-26T20:01:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a75600640 1 -- 172.21.15.88:0/1589757071 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68032810 con 0x7f1a78073790 2024-04-26T20:01:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 -- 172.21.15.88:0/1589757071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 msgr2=0x7f1a7806dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 --2- 172.21.15.88:0/1589757071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a7806dde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1a780d1d70 tx=0x7f1a6802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 -- 172.21.15.88:0/1589757071 shutdown_connections 2024-04-26T20:01:16.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 --2- 172.21.15.88:0/1589757071 >> v2:172.21.15.132:3300/0 conn(0x7f1a7815d550 0x7f1a780b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 --2- 172.21.15.88:0/1589757071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a7806dde0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 -- 172.21.15.88:0/1589757071 >> 172.21.15.88:0/1589757071 conn(0x7f1a780b2a30 msgr2=0x7f1a780b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 -- 172.21.15.88:0/1589757071 shutdown_connections 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.043+0000 7f1a7de00640 1 -- 172.21.15.88:0/1589757071 wait complete. 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.045+0000 7f1a7de00640 1 Processor -- start 2024-04-26T20:01:16.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.045+0000 7f1a7de00640 1 -- start start 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 0x7f1a7815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7815d550 0x7f1a7814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78074140 con 0x7f1a78073790 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78073fc0 con 0x7f1a78073b70 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a7de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a780adea0 con 0x7f1a7815d550 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a76a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a76a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58254/0 (socket says 172.21.15.88:58254) 2024-04-26T20:01:16.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.046+0000 7f1a76a00640 1 -- 172.21.15.88:0/716350456 learned_addr learned my addr 172.21.15.88:0/716350456 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76000640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 0x7f1a7815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a77400640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7815d550 0x7f1a7814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 -- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7815d550 msgr2=0x7f1a7814be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7815d550 0x7f1a7814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 -- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 msgr2=0x7f1a7815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 0x7f1a7815b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 -- 172.21.15.88:0/716350456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a78143150 con 0x7f1a78073790 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76000640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 0x7f1a7815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a76a00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000c00 tx=0x7f1a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.047+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68037040 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.048+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68004250 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.048+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a78143430 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.048+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68006e10 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.048+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a781439f0 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.049+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78069650 con 0x7f1a78073790 2024-04-26T20:01:16.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.050+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a68030770 con 0x7f1a78073790 2024-04-26T20:01:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.051+0000 7f1a6f400640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 0x7f1a5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.051+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1a680acda0 con 0x7f1a78073790 2024-04-26T20:01:16.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.051+0000 7f1a76000640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 0x7f1a5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.052+0000 7f1a76000640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 0x7f1a5806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006fd0 tx=0x7f1a64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.055+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a6807b250 con 0x7f1a78073790 2024-04-26T20:01:16.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.328+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1a7814dc40 con 0x7f1a78073790 2024-04-26T20:01:16.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.332+0000 7f1a6f400640 1 -- 172.21.15.88:0/716350456 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f1a6807ab50 con 0x7f1a78073790 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw== 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:01:16.333 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 msgr2=0x7f1a5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 0x7f1a5806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a64006fd0 tx=0x7f1a64008040 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 msgr2=0x7f1a780b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1a68000c00 tx=0x7f1a680046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 shutdown_connections 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a5806c920 0x7f1a5806ede0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a7815d550 0x7f1a7814be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a78073b70 0x7f1a7815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 --2- 172.21.15.88:0/716350456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a78073790 0x7f1a780b5740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 >> 172.21.15.88:0/716350456 conn(0x7f1a780b2a30 msgr2=0x7f1a78150dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 shutdown_connections 2024-04-26T20:01:16.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.335+0000 7f1a7de00640 1 -- 172.21.15.88:0/716350456 wait complete. 2024-04-26T20:01:16.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:16.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.539+0000 7fbd23200640 1 Processor -- start 2024-04-26T20:01:16.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.539+0000 7fbd23200640 1 -- start start 2024-04-26T20:01:16.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd23200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd2415f1b0 0x7fbd240d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd23200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd23200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd240d3f50 con 0x7fbd24162b00 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd23200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fbd240d36b0 con 0x7fbd2415f1b0 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd23200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbd240d3990 con 0x7fbd24163650 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd2415f1b0 0x7fbd240d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fbd2415f1b0 0x7fbd240d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58258/0 (socket says 172.21.15.88:58258) 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 -- 172.21.15.88:0/1468000081 learned_addr learned my addr 172.21.15.88:0/1468000081 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:16.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 -- 172.21.15.88:0/1468000081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 msgr2=0x7fbd240b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 --2- 172.21.15.88:0/1468000081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 -- 172.21.15.88:0/1468000081 >> v1:172.21.15.132:6789/0 conn(0x7fbd24162b00 legacy=0x7fbd24162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.541+0000 7fbd21e00640 1 -- 172.21.15.88:0/1468000081 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd240d3c70 con 0x7fbd2415f1b0 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.542+0000 7fbd21e00640 1 --2- 172.21.15.88:0/1468000081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240d5990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009870 tx=0x7fbd1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f903055686b21d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd20a00640 1 -- 172.21.15.88:0/1468000081 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14037070 con 0x7fbd2415f1b0 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd20a00640 1 -- 172.21.15.88:0/1468000081 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd14032440 con 0x7fbd2415f1b0 2024-04-26T20:01:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd20a00640 1 -- 172.21.15.88:0/1468000081 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14032810 con 0x7fbd2415f1b0 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 -- 172.21.15.88:0/1468000081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 msgr2=0x7fbd240d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 --2- 172.21.15.88:0/1468000081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240d5990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009870 tx=0x7fbd1402a210 comp rx=0 tx=0).stop 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 -- 172.21.15.88:0/1468000081 shutdown_connections 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 --2- 172.21.15.88:0/1468000081 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 --2- 172.21.15.88:0/1468000081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240d5990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 -- 172.21.15.88:0/1468000081 >> 172.21.15.88:0/1468000081 conn(0x7fbd2409e230 msgr2=0x7fbd240d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:16.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 -- 172.21.15.88:0/1468000081 shutdown_connections 2024-04-26T20:01:16.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.543+0000 7fbd23200640 1 -- 172.21.15.88:0/1468000081 wait complete. 2024-04-26T20:01:16.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.545+0000 7fbd23200640 1 Processor -- start 2024-04-26T20:01:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.545+0000 7fbd23200640 1 -- start start 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd24162b00 0x7fbd24151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240b87f0 con 0x7fbd2415f1b0 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240d33a0 con 0x7fbd24162b00 2024-04-26T20:01:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd23200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbd240b8970 con 0x7fbd24163650 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd21e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd21e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58264/0 (socket says 172.21.15.88:58264) 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.546+0000 7fbd21e00640 1 -- 172.21.15.88:0/1926513217 learned_addr learned my addr 172.21.15.88:0/1926513217 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 -- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 msgr2=0x7fbd240cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 -- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd24162b00 msgr2=0x7fbd24151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd24162b00 0x7fbd24151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 -- 172.21.15.88:0/1926513217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd241f5f90 con 0x7fbd2415f1b0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd21e00640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009870 tx=0x7fbd140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14041070 con 0x7fbd2415f1b0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1403c870 con 0x7fbd2415f1b0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd241f6250 con 0x7fbd2415f1b0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.547+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14037040 con 0x7fbd2415f1b0 2024-04-26T20:01:16.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.548+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241f6810 con 0x7fbd2415f1b0 2024-04-26T20:01:16.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.549+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd240693c0 con 0x7fbd2415f1b0 2024-04-26T20:01:16.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.549+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd14044070 con 0x7fbd2415f1b0 2024-04-26T20:01:16.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.550+0000 7fbd1a000640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 0x7fbd0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:16.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.550+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbd140ac5b0 con 0x7fbd2415f1b0 2024-04-26T20:01:16.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.551+0000 7fbd21400640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 0x7fbd0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:16.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.551+0000 7fbd21400640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 0x7fbd0406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd240d5740 tx=0x7fbd10007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:16.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.554+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd1407aa70 con 0x7fbd2415f1b0 2024-04-26T20:01:16.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.791+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd241f69e0 con 0x7fbd2415f1b0 2024-04-26T20:01:16.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.792+0000 7fbd1a000640 1 -- 172.21.15.88:0/1926513217 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5184 (secure 0 0 0) 0x7fbd1407a370 con 0x7fbd2415f1b0 2024-04-26T20:01:16.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:16.793 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4705,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/2458488289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2458488289},{"type":"v1","addr":"172.21.15.88:6837","nonce":2458488289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:16.794 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 15 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 msgr2=0x7fbd0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 0x7fbd0406ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd240d5740 tx=0x7fbd10007990 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 msgr2=0x7fbd240ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009870 tx=0x7fbd140046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 shutdown_connections 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbd0406c920 0x7fbd0406ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbd24163650 0x7fbd240cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd24162b00 0x7fbd24151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 --2- 172.21.15.88:0/1926513217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd2415f1b0 0x7fbd240ce6f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 >> 172.21.15.88:0/1926513217 conn(0x7fbd2409e230 msgr2=0x7fbd240b8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 shutdown_connections 2024-04-26T20:01:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:16.795+0000 7fbd23200640 1 -- 172.21.15.88:0/1926513217 wait complete. 2024-04-26T20:01:17.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:01:05.341746+0000', 'modified': '2024-04-26T20:01:08.420438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4708': {'gid': 4708, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.88:6836/351605980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 351605980}, {'type': 'v1', 'addr': '172.21.15.88:6836', 'nonce': 351605980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4708, 'qdb_cluster': [4708]} 2024-04-26T20:01:17.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:01:17.218 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:01:17.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.425+0000 7f27c7e00640 1 Processor -- start 2024-04-26T20:01:17.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.425+0000 7f27c7e00640 1 -- start start 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27c80d0e70 0x7f27c80d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f27c80b5d40 0x7f27c80aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27c80d1790 con 0x7f27c80b5660 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c7e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f27c80727f0 con 0x7f27c80d0e70 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f27c80b4a10 con 0x7f27c80b5d40 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27c80d0e70 0x7f27c80d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c6a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f27c80d0e70 0x7f27c80d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58266/0 (socket says 172.21.15.88:58266) 2024-04-26T20:01:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.427+0000 7f27c6a00640 1 -- 172.21.15.88:0/3791271657 learned_addr learned my addr 172.21.15.88:0/3791271657 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.428+0000 7f27c7400640 1 --2- 172.21.15.88:0/3791271657 >> v2:172.21.15.132:3300/0 conn(0x7f27c80b5d40 0x7f27c80aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.428+0000 7f27c6a00640 1 -- 172.21.15.88:0/3791271657 >> v2:172.21.15.132:3300/0 conn(0x7f27c80b5d40 msgr2=0x7f27c80aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.428+0000 7f27c6a00640 1 --2- 172.21.15.88:0/3791271657 >> v2:172.21.15.132:3300/0 conn(0x7f27c80b5d40 0x7f27c80aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.428+0000 7f27c6a00640 1 -- 172.21.15.88:0/3791271657 >> v1:172.21.15.132:6789/0 conn(0x7f27c80b5660 legacy=0x7f27c80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.428+0000 7f27c6a00640 1 -- 172.21.15.88:0/3791271657 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c80b4cf0 con 0x7f27c80d0e70 2024-04-26T20:01:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.429+0000 7f27c6a00640 1 --2- 172.21.15.88:0/3791271657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c80d1250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27c80720e0 tx=0x7f27b802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1b6e5113630e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.430+0000 7f27c6000640 1 -- 172.21.15.88:0/3791271657 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8037070 con 0x7f27c80d0e70 2024-04-26T20:01:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.430+0000 7f27c6000640 1 -- 172.21.15.88:0/3791271657 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b8032440 con 0x7f27c80d0e70 2024-04-26T20:01:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.430+0000 7f27c6000640 1 -- 172.21.15.88:0/3791271657 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8032810 con 0x7f27c80d0e70 2024-04-26T20:01:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 -- 172.21.15.88:0/3791271657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 msgr2=0x7f27c80d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 --2- 172.21.15.88:0/3791271657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c80d1250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27c80720e0 tx=0x7f27b802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 -- 172.21.15.88:0/3791271657 shutdown_connections 2024-04-26T20:01:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 --2- 172.21.15.88:0/3791271657 >> v2:172.21.15.132:3300/0 conn(0x7f27c80b5d40 0x7f27c80aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 --2- 172.21.15.88:0/3791271657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c80d1250 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 -- 172.21.15.88:0/3791271657 >> 172.21.15.88:0/3791271657 conn(0x7f27c806c740 msgr2=0x7f27c806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 -- 172.21.15.88:0/3791271657 shutdown_connections 2024-04-26T20:01:17.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.431+0000 7f27c7e00640 1 -- 172.21.15.88:0/3791271657 wait complete. 2024-04-26T20:01:17.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.433+0000 7f27c7e00640 1 Processor -- start 2024-04-26T20:01:17.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.433+0000 7f27c7e00640 1 -- start start 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27c80b5660 0x7f27c8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27c80b5d40 0x7f27c81618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27c80b05b0 con 0x7f27c80d0e70 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f27c80b0430 con 0x7f27c80b5d40 2024-04-26T20:01:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.434+0000 7f27c7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f27c815f1a0 con 0x7f27c80b5660 2024-04-26T20:01:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58270/0 (socket says 172.21.15.88:58270) 2024-04-26T20:01:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 -- 172.21.15.88:0/817429951 learned_addr learned my addr 172.21.15.88:0/817429951 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c6a00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27c80b5660 0x7f27c8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27bfe00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27c80b5d40 0x7f27c81618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 -- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27c80b5660 msgr2=0x7f27c8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27c80b5660 0x7f27c8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 -- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27c80b5d40 msgr2=0x7f27c81618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27c80b5d40 0x7f27c81618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 -- 172.21.15.88:0/817429951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c8153a30 con 0x7f27c80d0e70 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.435+0000 7f27c7400640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007e40 tx=0x7f27c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.436+0000 7f27bf400640 1 -- 172.21.15.88:0/817429951 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0014620 con 0x7f27c80d0e70 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.436+0000 7f27bf400640 1 -- 172.21.15.88:0/817429951 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c0019070 con 0x7f27c80d0e70 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.436+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c8153d10 con 0x7f27c80d0e70 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.436+0000 7f27bf400640 1 -- 172.21.15.88:0/817429951 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0014a20 con 0x7f27c80d0e70 2024-04-26T20:01:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.436+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81542d0 con 0x7f27c80d0e70 2024-04-26T20:01:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.438+0000 7f27bf400640 1 -- 172.21.15.88:0/817429951 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f27c0014bc0 con 0x7f27c80d0e70 2024-04-26T20:01:17.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.439+0000 7f27bf400640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 0x7f27a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.439+0000 7f27bf400640 1 -- 172.21.15.88:0/817429951 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f27c008c4b0 con 0x7f27c80d0e70 2024-04-26T20:01:17.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.439+0000 7f27c6a00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 0x7f27a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.440+0000 7f27d0c79740 1 Processor -- start 2024-04-26T20:01:17.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.440+0000 7f27d0c79740 1 -- start start 2024-04-26T20:01:17.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.440+0000 7f27c6a00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 0x7f27a406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f27b8034000 tx=0x7f27b802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f521b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a95f485600 con 0x55a95f5217b0 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a95f422100 con 0x55a95f55de80 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a95f4815c0 con 0x55a95f54c270 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58276/0 (socket says 172.21.15.88:58276) 2024-04-26T20:01:17.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 -- 172.21.15.88:0/1552532790 learned_addr learned my addr 172.21.15.88:0/1552532790 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 -- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 msgr2=0x55a95f54c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 -- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 msgr2=0x55a95f521b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f521b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.441+0000 7f27c6a00640 1 -- 172.21.15.88:0/1552532790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95f1972d0 con 0x55a95f5217b0 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27b802a790 tx=0x7f27b802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293a2423ab4d4a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27b7400640 1 -- 172.21.15.88:0/1552532790 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8037040 con 0x55a95f5217b0 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27b7400640 1 -- 172.21.15.88:0/1552532790 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b803e460 con 0x55a95f5217b0 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27b7400640 1 -- 172.21.15.88:0/1552532790 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b803e810 con 0x55a95f5217b0 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27d0c79740 1 -- 172.21.15.88:0/1552532790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 msgr2=0x55a95f523910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.442+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f27b802a790 tx=0x7f27b802c000 comp rx=0 tx=0).stop 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 -- 172.21.15.88:0/1552532790 shutdown_connections 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f521b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1552532790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f523910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 -- 172.21.15.88:0/1552532790 >> 172.21.15.88:0/1552532790 conn(0x55a95f4b5d10 msgr2=0x55a95f567270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 -- 172.21.15.88:0/1552532790 shutdown_connections 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 -- 172.21.15.88:0/1552532790 wait complete. 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.443+0000 7f27d0c79740 1 -- start start 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 Processor -- start 2024-04-26T20:01:17.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 0x55a95f53c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 0x55a95f54e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a95f40cf70 con 0x55a95f5217b0 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.444+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a95f549dc0 con 0x55a95f54c270 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a95f54dc80 con 0x55a95f55de80 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27d0c79740 10 client.0 _ll_register_callbacks cb 0x55a95f47b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58280/0 (socket says 172.21.15.88:58280) 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 -- 172.21.15.88:0/1443590108 learned_addr learned my addr 172.21.15.88:0/1443590108 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27bfe00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 0x55a95f53c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c7400640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 0x55a95f54e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 -- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 msgr2=0x55a95f54e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 0x55a95f54e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 -- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 msgr2=0x55a95f53c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 0x55a95f53c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c6a00640 1 -- 172.21.15.88:0/1443590108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95f2616b0 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27c7400640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 0x55a95f54e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.445+0000 7f27bfe00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 0x55a95f53c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27c6a00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009630 tx=0x7f27b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6987a372ea030f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27abe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8037040 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f27b803ed50 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b803e460 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f27b805b050 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27d0c79740 10 client.5415 fetch_fsmap learned FSMap version 15 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55a95f54e940 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.446+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b803e460 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f27b805e020 con 0x55a95f5217b0 2024-04-26T20:01:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 10 client.5415 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 10 client.5415 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 20 client.5415 populate_metadata read hostname 'smithi088' 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f582740 0x55a95f584b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 4 client.5415 mds_command: new command op to 4708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 --> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a95f493210 con 0x55a95f582740 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.447+0000 7f27bfe00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f582740 0x55a95f584b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.448+0000 7f27abe00640 10 client.5415 ms_handle_connect on v2:172.21.15.88:6834/351605980 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.448+0000 7f27bfe00640 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f582740 0x55a95f584b30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2983fc100a23ba82 server_cookie=e67dc7067d3799f1 in_seq=0 out_seq=0 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.448+0000 7f27abe00640 1 -- 172.21.15.88:0/1443590108 <== mds.0 v2:172.21.15.88:6834/351605980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a95f493210 con 0x55a95f582740 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.448+0000 7f27abe00640 10 client.5415 handle_command_reply: tid=0 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 client.5415 shutdown 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 msgr2=0x55a95f560d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009630 tx=0x7f27b8004730 comp rx=0 tx=0).stop 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 shutdown_connections 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f582740 0x55a95f584b30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f55de80 0x55a95f54e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f54c270 0x55a95f53c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 --2- 172.21.15.88:0/1443590108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f560d90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27abe00640 0 client.5415 ms_handle_reset on v2:172.21.15.88:6834/351605980 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 >> 172.21.15.88:0/1443590108 conn(0x55a95f4b5d10 msgr2=0x55a95f549f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 shutdown_connections 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 1 -- 172.21.15.88:0/1443590108 wait complete. 2024-04-26T20:01:17.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.449+0000 7f27d0c79740 20 client.5415 trim_cache size 0 max 16384 2024-04-26T20:01:17.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.469+0000 7f27d0c79740 1 Processor -- start 2024-04-26T20:01:17.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.469+0000 7f27d0c79740 1 -- start start 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.469+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f54e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27d0c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27c82006d0 con 0x55a95f5217b0 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f27c81604a0 con 0x55a95f55de80 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f27c8160af0 con 0x55a95f54c270 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58296/0 (socket says 172.21.15.88:58296) 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 -- 172.21.15.88:0/2494714946 learned_addr learned my addr 172.21.15.88:0/2494714946 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 -- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 msgr2=0x55a95f54dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 -- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 msgr2=0x55a95f54e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c7400640 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f54e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f54e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.470+0000 7f27c6a00640 1 -- 172.21.15.88:0/2494714946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a95f51c840 con 0x55a95f5217b0 2024-04-26T20:01:17.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27c6a00640 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27b8032cf0 tx=0x7f27b804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351202b98e9a6fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27b6a00640 1 -- 172.21.15.88:0/2494714946 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8037040 con 0x55a95f5217b0 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27b6a00640 1 -- 172.21.15.88:0/2494714946 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b803c7b0 con 0x55a95f5217b0 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27b6a00640 1 -- 172.21.15.88:0/2494714946 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b803cb60 con 0x55a95f5217b0 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 -- 172.21.15.88:0/2494714946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 msgr2=0x55a95f5674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f27b8032cf0 tx=0x7f27b804c000 comp rx=0 tx=0).stop 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 -- 172.21.15.88:0/2494714946 shutdown_connections 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f54e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f54c270 0x55a95f54dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2494714946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f5217b0 0x55a95f5674a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.471+0000 7f27d0c79740 1 -- 172.21.15.88:0/2494714946 >> 172.21.15.88:0/2494714946 conn(0x55a95f4b5d10 msgr2=0x55a95f517df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.472+0000 7f27d0c79740 1 -- 172.21.15.88:0/2494714946 shutdown_connections 2024-04-26T20:01:17.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.472+0000 7f27d0c79740 1 -- 172.21.15.88:0/2494714946 wait complete. 2024-04-26T20:01:17.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.472+0000 7f27d0c79740 1 -- start start 2024-04-26T20:01:17.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.472+0000 7f27d0c79740 1 Processor -- start 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f5217b0 0x55a95f5e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f55cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a95f40cf70 con 0x55a95f54c270 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a95f549dc0 con 0x55a95f55de80 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.473+0000 7f27d0c79740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a95f5808d0 con 0x55a95f5217b0 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27d0c79740 10 client.0 _ll_register_callbacks cb 0x7f27b80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58310/0 (socket says 172.21.15.88:58310) 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 -- 172.21.15.88:0/2149720685 learned_addr learned my addr 172.21.15.88:0/2149720685 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 -- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f5217b0 msgr2=0x55a95f5e0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f5217b0 0x55a95f5e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 -- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 msgr2=0x55a95f55cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f55cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 -- 172.21.15.88:0/2149720685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a95f548f90 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27bfe00640 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f27c8155630 tx=0x7f27b0005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21407f901dec3508 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.474+0000 7f27abe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0010040 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f27b0016ca0 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b803c7b0 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f27b0029050 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 10 client.5421 fetch_fsmap learned FSMap version 15 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55a95f53bec0 con 0x55a95f54c270 2024-04-26T20:01:17.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b803c7b0 con 0x55a95f54c270 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f27b002c020 con 0x55a95f54c270 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 10 client.5421 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 10 client.5421 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 20 client.5421 populate_metadata read hostname 'smithi088' 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f56a3b0 0x55a95f56a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 4 client.5421 mds_command: new command op to 4708 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.475+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 --> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] -- command(tid 0: {"prefix": "status"}) -- 0x55a95f48d080 con 0x55a95f56a3b0 2024-04-26T20:01:17.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27c6a00640 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f56a3b0 0x55a95f56a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27abe00640 10 client.5421 ms_handle_connect on v2:172.21.15.88:6834/351605980 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27c6a00640 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f56a3b0 0x55a95f56a790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2184c78ba27eb681 server_cookie=7941006bd00327d7 in_seq=0 out_seq=0 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27abe00640 1 -- 172.21.15.88:0/2149720685 <== mds.0 v2:172.21.15.88:6834/351605980 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55a95f48d080 con 0x55a95f56a3b0 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27abe00640 10 client.5421 handle_command_reply: tid=0 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27d0c79740 1 client.5421 shutdown 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 msgr2=0x55a95f53b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.476+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f27c8155630 tx=0x7f27b0005330 comp rx=0 tx=0).stop 2024-04-26T20:01:17.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 shutdown_connections 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:6834/351605980,v1:172.21.15.88:6836/351605980] conn(0x55a95f56a3b0 0x55a95f56a790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a95f55de80 0x55a95f55cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a95f54c270 0x55a95f53b850 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 --2- 172.21.15.88:0/2149720685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a95f5217b0 0x55a95f5e0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27abe00640 0 client.5421 ms_handle_reset on v2:172.21.15.88:6834/351605980 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 >> 172.21.15.88:0/2149720685 conn(0x55a95f4b5d10 msgr2=0x55a95f51b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 shutdown_connections 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 1 -- 172.21.15.88:0/2149720685 wait complete. 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27d0c79740 20 client.5421 trim_cache size 0 max 16384 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "100f2996-174e-4373-bd56-473fe683ba7a", 2024-04-26T20:01:17.480 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4708, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.557776294, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 15, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 33, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 33, 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 20.449116423 2024-04-26T20:01:17.481 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 msgr2=0x7f27a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.477+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 0x7f27a406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f27b8034000 tx=0x7f27b802e000 comp rx=0 tx=0).stop 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 msgr2=0x7f27c8161e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f27c0007e40 tx=0x7f27c000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 shutdown_connections 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27c80d0e70 0x7f27c8161e30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27c80b5d40 0x7f27c81618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f27a406c8f0 0x7f27a406edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 --2- 172.21.15.88:0/817429951 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f27c80b5660 0x7f27c8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 >> 172.21.15.88:0/817429951 conn(0x7f27c806c740 msgr2=0x7f27c80b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 shutdown_connections 2024-04-26T20:01:17.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.478+0000 7f27c7e00640 1 -- 172.21.15.88:0/817429951 wait complete. 2024-04-26T20:01:17.496 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:01:17.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.683+0000 7faf09e00640 1 Processor -- start 2024-04-26T20:01:17.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.683+0000 7faf09e00640 1 -- start start 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faf09e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7faf040cd650 0x7faf040add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faf09e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf040af9a0 con 0x7faf0406ce80 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faf09e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf040afc80 con 0x7faf0415cdf0 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faf09e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7faf040ae2c0 con 0x7faf040cd650 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faefa000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7faf0406ce80 0x7faf040b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:34380/0 (socket says 172.21.15.88:34380) 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faefa000640 1 -- 172.21.15.88:0/4168990274 learned_addr learned my addr 172.21.15.88:0/4168990274 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:17.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.685+0000 7faf03400640 1 --2- 172.21.15.88:0/4168990274 >> v2:172.21.15.88:3300/0 conn(0x7faf040cd650 0x7faf040add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.686+0000 7faf02000640 1 -- 172.21.15.88:0/4168990274 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544445358 0 0) 0x7faf040af9a0 con 0x7faf0406ce80 2024-04-26T20:01:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.686+0000 7faf02000640 1 -- 172.21.15.88:0/4168990274 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faee8003520 con 0x7faf0406ce80 2024-04-26T20:01:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.686+0000 7faf03400640 1 -- 172.21.15.88:0/4168990274 >> v1:172.21.15.132:6789/0 conn(0x7faf0415cdf0 legacy=0x7faf0406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.686+0000 7faf03400640 1 -- 172.21.15.88:0/4168990274 >> v1:172.21.15.88:6789/0 conn(0x7faf0406ce80 legacy=0x7faf040b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.686+0000 7faf03400640 1 -- 172.21.15.88:0/4168990274 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf040aff60 con 0x7faf040cd650 2024-04-26T20:01:17.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.687+0000 7faf03400640 1 --2- 172.21.15.88:0/4168990274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040add80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097a0 tx=0x7faefc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca5173622d0c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.687+0000 7faf02000640 1 -- 172.21.15.88:0/4168990274 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc037070 con 0x7faf040cd650 2024-04-26T20:01:17.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.687+0000 7faf02000640 1 -- 172.21.15.88:0/4168990274 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faefc032440 con 0x7faf040cd650 2024-04-26T20:01:17.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.687+0000 7faf02000640 1 -- 172.21.15.88:0/4168990274 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc032810 con 0x7faf040cd650 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 -- 172.21.15.88:0/4168990274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 msgr2=0x7faf040add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 --2- 172.21.15.88:0/4168990274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040add80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097a0 tx=0x7faefc02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 -- 172.21.15.88:0/4168990274 shutdown_connections 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 --2- 172.21.15.88:0/4168990274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040add80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 -- 172.21.15.88:0/4168990274 >> 172.21.15.88:0/4168990274 conn(0x7faf040b2d50 msgr2=0x7faf040b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 -- 172.21.15.88:0/4168990274 shutdown_connections 2024-04-26T20:01:17.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.688+0000 7faf09e00640 1 -- 172.21.15.88:0/4168990274 wait complete. 2024-04-26T20:01:17.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.689+0000 7faf09e00640 1 Processor -- start 2024-04-26T20:01:17.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.689+0000 7faf09e00640 1 -- start start 2024-04-26T20:01:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf0406ce80 0x7faf0414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf0415cdf0 0x7faf0415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf0415d1d0 con 0x7faf040cd650 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faf040b05b0 con 0x7faf0415cdf0 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.690+0000 7faf09e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf0415d350 con 0x7faf0406ce80 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58322/0 (socket says 172.21.15.88:58322) 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 -- 172.21.15.88:0/2558315672 learned_addr learned my addr 172.21.15.88:0/2558315672 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 -- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf0406ce80 msgr2=0x7faf0414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf0406ce80 0x7faf0414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faf03400640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf0415cdf0 0x7faf0415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 -- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf0415cdf0 msgr2=0x7faf0415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf0415cdf0 0x7faf0415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 -- 172.21.15.88:0/2558315672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0415d5d0 con 0x7faf040cd650 2024-04-26T20:01:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefa000640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c430 tx=0x7faeec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.691+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec017070 con 0x7faf040cd650 2024-04-26T20:01:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.692+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec005ce0 con 0x7faf040cd650 2024-04-26T20:01:17.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.692+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0072f0 con 0x7faf040cd650 2024-04-26T20:01:17.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.692+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0415d8b0 con 0x7faf040cd650 2024-04-26T20:01:17.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.692+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0414bc70 con 0x7faf040cd650 2024-04-26T20:01:17.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.693+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf04069540 con 0x7faf040cd650 2024-04-26T20:01:17.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.694+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeec01a070 con 0x7faf040cd650 2024-04-26T20:01:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.694+0000 7faefbe00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.695+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faeec08b0d0 con 0x7faf040cd650 2024-04-26T20:01:17.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.695+0000 7faf02a00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:17.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.695+0000 7faf02a00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab90 tx=0x7faef4009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.699+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faeec059460 con 0x7faf040cd650 2024-04-26T20:01:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.922+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faf040d0140 con 0x7faf040cd650 2024-04-26T20:01:17.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.923+0000 7faefbe00640 1 -- 172.21.15.88:0/2558315672 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7faeec058d60 con 0x7faf040cd650 2024-04-26T20:01:17.924 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:01:17.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.925+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 msgr2=0x7faee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.925+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab90 tx=0x7faef4009230 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.925+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 msgr2=0x7faf040cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.925+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c430 tx=0x7faeec00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 shutdown_connections 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf0415cdf0 0x7faf0415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf040cd650 0x7faf040cfc00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 --2- 172.21.15.88:0/2558315672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf0406ce80 0x7faf0414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 >> 172.21.15.88:0/2558315672 conn(0x7faf040b2d50 msgr2=0x7faf040d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 shutdown_connections 2024-04-26T20:01:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:17.926+0000 7faf09e00640 1 -- 172.21.15.88:0/2558315672 wait complete. 2024-04-26T20:01:17.944 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:01:17.945 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:01:17.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:17.945 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-04-26T20:01:17.996 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:01:17.997 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42499sec preferred_lft 42499sec 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::9c6e:acff:fec3:49a/64 scope link 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:33:4d:d7:93:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c433:4dff:fed7:93b6/64 scope link 2024-04-26T20:01:17.998 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.999 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:01:17.999 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:01:17.999 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::2caf:4ff:fe1f:b3fa/64 scope link 2024-04-26T20:01:17.999 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:17.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:17.999 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:01:18.076 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-04-26T20:01:18.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:18.080 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-04-26T20:01:18.132 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-04-26T20:01:18.132 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:01:18.132 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:01:18.132 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:01:18.132 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:01:18.133 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:01:18.133 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:01:18.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:01:18.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:01:18.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:01:18.134 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:01:18.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:01:18.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:01:18.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:18.336 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:01:18.384 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:01:18.384 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:01:18.385 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:01:18.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:18.385 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-04-26T20:01:18.467 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:01:18.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:01:18.467 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:18.521 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:01:18.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:01:18.521 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:18.571 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:01:18.655 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:01:18.710 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:01:18.711 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:01:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:01:18.713 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:01:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:18.714 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:01:18.764 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:01:18.765 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:01:18.769 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:01:18.834 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:01:18.874 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:01:18.874 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:01:18.875 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:01:18.876 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:01:18.877 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:01:18.877 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:01:18.877 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:01:18.877 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:01:18.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:18.878 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:01:18.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2024-04-26T20:01:18.930+0000 7f251c398480 -1 init, newargv = 0x55d59eeac0b0 newargc=15 2024-04-26T20:01:18.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[47131]: starting ceph client 2024-04-26T20:01:18.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[47131]: starting fuse 2024-04-26T20:01:18.960 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:01:18.960 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:01:18.960 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:01:18.960 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-04-26T20:01:18.961 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2024-04-26T20:01:18.962 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-04-26T20:01:19.150 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.47131.asok 2024-04-26T20:01:19.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:01:19.167 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47131.asok 2024-04-26T20:01:19.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:19.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47131.asok status 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "47131", 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:19.398 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5433, 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5433 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 97307219 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:01:19.399 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 97307219 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.5433 192.168.0.1:0/97307219", 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/97307219", 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 15, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 33, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-04-26T20:01:19.400 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:01:19.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:19.413 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-04-26T20:01:19.413 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:19.467 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-04-26T20:01:19.467 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:19.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:19.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:19.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:01:19.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.743+0000 7fe014e00640 1 Processor -- start 2024-04-26T20:01:19.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.743+0000 7fe014e00640 1 -- start start 2024-04-26T20:01:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.744+0000 7fe014e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe01006c770 0x7fe01006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.744+0000 7fe014e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe014e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01006d090 con 0x7fe01015cfd0 2024-04-26T20:01:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe014e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100af460 con 0x7fe01006c770 2024-04-26T20:01:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe014e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe0100ae850 con 0x7fe01015cb70 2024-04-26T20:01:19.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe00de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe01006c770 0x7fe01006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe005400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe00de00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe01006c770 0x7fe01006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:38932/0 (socket says 172.21.15.88:38932) 2024-04-26T20:01:19.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe005400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:33640/0 (socket says 172.21.15.88:33640) 2024-04-26T20:01:19.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.745+0000 7fe00de00640 1 -- 172.21.15.88:0/1848956258 learned_addr learned my addr 172.21.15.88:0/1848956258 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.746+0000 7fe00de00640 1 -- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 msgr2=0x7fe010073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.746+0000 7fe00de00640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.746+0000 7fe00de00640 1 -- 172.21.15.88:0/1848956258 >> v1:172.21.15.132:6789/0 conn(0x7fe01015cfd0 legacy=0x7fe010073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.746+0000 7fe00de00640 1 -- 172.21.15.88:0/1848956258 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100aeb30 con 0x7fe01006c770 2024-04-26T20:01:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.746+0000 7fe005400640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:01:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.747+0000 7fe00de00640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01006c770 0x7fe01006cb50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe00000bb10 tx=0x7fe00002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca5743a924e3981 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.748+0000 7fe00d400640 1 -- 172.21.15.88:0/1848956258 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000036020 con 0x7fe01006c770 2024-04-26T20:01:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.748+0000 7fe00d400640 1 -- 172.21.15.88:0/1848956258 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe000032820 con 0x7fe01006c770 2024-04-26T20:01:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.748+0000 7fe00d400640 1 -- 172.21.15.88:0/1848956258 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000032bf0 con 0x7fe01006c770 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.748+0000 7fe014e00640 1 -- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01006c770 msgr2=0x7fe01006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.748+0000 7fe014e00640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01006c770 0x7fe01006cb50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe00000bb10 tx=0x7fe00002c920 comp rx=0 tx=0).stop 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 -- 172.21.15.88:0/1848956258 shutdown_connections 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01015cb70 0x7fe010073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 --2- 172.21.15.88:0/1848956258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01006c770 0x7fe01006cb50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 -- 172.21.15.88:0/1848956258 >> 172.21.15.88:0/1848956258 conn(0x7fe0100b2d50 msgr2=0x7fe0100b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 -- 172.21.15.88:0/1848956258 shutdown_connections 2024-04-26T20:01:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.749+0000 7fe014e00640 1 -- 172.21.15.88:0/1848956258 wait complete. 2024-04-26T20:01:19.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.750+0000 7fe014e00640 1 Processor -- start 2024-04-26T20:01:19.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.751+0000 7fe014e00640 1 -- start start 2024-04-26T20:01:19.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.751+0000 7fe014e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01006c770 0x7fe01015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.751+0000 7fe014e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01015cb70 0x7fe01009c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.751+0000 7fe014e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.751+0000 7fe014e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe01015c600 con 0x7fe01015cfd0 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe014e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100af0f0 con 0x7fe01015cb70 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe014e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe01015c780 con 0x7fe01006c770 2024-04-26T20:01:19.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42642/0 (socket says 172.21.15.88:42642) 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 -- 172.21.15.88:0/1062059004 learned_addr learned my addr 172.21.15.88:0/1062059004 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00de00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01006c770 0x7fe01015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe005400640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01015cb70 0x7fe01009c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 -- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01006c770 msgr2=0x7fe01015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01006c770 0x7fe01015ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 -- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01015cb70 msgr2=0x7fe01009c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01015cb70 0x7fe01009c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.752+0000 7fe00e800640 1 -- 172.21.15.88:0/1062059004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100ca4c0 con 0x7fe01015cfd0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe00e800640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c6b0 tx=0x7fe00800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008003b70 con 0x7fe01015cfd0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe008003d30 con 0x7fe01015cfd0 2024-04-26T20:01:19.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008005310 con 0x7fe01015cfd0 2024-04-26T20:01:19.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01009d090 con 0x7fe01015cfd0 2024-04-26T20:01:19.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.753+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01009d930 con 0x7fe01015cfd0 2024-04-26T20:01:19.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.754+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01009d480 con 0x7fe01015cfd0 2024-04-26T20:01:19.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.756+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0080054b0 con 0x7fe01015cfd0 2024-04-26T20:01:19.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.756+0000 7fe006a00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 0x7fdfec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:19.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.757+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe00808b9c0 con 0x7fe01015cfd0 2024-04-26T20:01:19.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.757+0000 7fe00de00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 0x7fdfec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:19.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.757+0000 7fe00de00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 0x7fdfec06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe000037000 tx=0x7fe000008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:19.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:19.761+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe008059de0 con 0x7fe01015cfd0 2024-04-26T20:01:20.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.005+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0100d1a40 con 0x7fe01015cfd0 2024-04-26T20:01:20.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.006+0000 7fe006a00640 1 -- 172.21.15.88:0/1062059004 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7fe0100d1a40 con 0x7fe01015cfd0 2024-04-26T20:01:20.008 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 msgr2=0x7fdfec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 0x7fdfec06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe000037000 tx=0x7fe000008040 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 msgr2=0x7fe01009c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c6b0 tx=0x7fe00800cb70 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 shutdown_connections 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe01015cfd0 0x7fe01009c9a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe01015cb70 0x7fe01009c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdfec06c8f0 0x7fdfec06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 --2- 172.21.15.88:0/1062059004 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe01006c770 0x7fe01015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 >> 172.21.15.88:0/1062059004 conn(0x7fe0100b2d50 msgr2=0x7fe01015f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 shutdown_connections 2024-04-26T20:01:20.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:20.009+0000 7fe014e00640 1 -- 172.21.15.88:0/1062059004 wait complete. 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:01:20.030 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:01:20.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.031 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.090 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42512sec preferred_lft 42512sec 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.091 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ccda:b9ff:fed8:349c/64 scope link 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::8029:7dff:fe56:31fa/64 scope link 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 92:6d:d7:ca:c5:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::906d:d7ff:feca:c5f7/64 scope link 2024-04-26T20:01:20.092 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:01:20.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.094 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:01:20.169 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T20:01:20.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.173 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-26T20:01:20.223 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-26T20:01:20.223 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:01:20.223 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:01:20.223 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:01:20.224 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:01:20.224 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:01:20.330 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:01:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:01:20.427 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:01:20.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.437 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:01:20.485 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:01:20.486 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:01:20.487 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:01:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.487 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-26T20:01:20.573 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:01:20.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:01:20.574 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:20.626 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:01:20.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:01:20.626 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:20.677 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:01:20.753 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:01:20.803 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:01:20.804 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:01:20.805 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:01:20.806 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:01:20.806 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:01:20.806 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:01:20.806 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:01:20.806 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:01:20.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.807 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:01:20.857 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:01:20.858 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:01:20.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:01:20.933 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:01:20.974 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:01:20.975 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:01:20.976 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:01:20.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:20.977 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:01:21.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:2024-04-26T20:01:21.031+0000 7f927bf20480 -1 init, newargv = 0x5577398e19e0 newargc=15 2024-04-26T20:01:21.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40036]: starting ceph client 2024-04-26T20:01:21.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40036]: starting fuse 2024-04-26T20:01:21.061 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:01:21.061 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:01:21.062 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.1")) 2024-04-26T20:01:21.063 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-04-26T20:01:21.244 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.1.40036.asok 2024-04-26T20:01:21.248 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:01:21.260 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40036.asok 2024-04-26T20:01:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:21.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40036.asok status 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "1", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "40036", 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-04-26T20:01:21.487 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 4634, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 4634 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3117959940 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:01:21.488 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3117959940 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.4634 192.168.0.1:0/3117959940", 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.1:0/3117959940", 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 15, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 33, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-04-26T20:01:21.489 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-26T20:01:21.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:21.502 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-04-26T20:01:21.502 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:21.554 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-04-26T20:01:21.554 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:21.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:21.555 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:21.636 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T20:01:21.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.851+0000 7fd773c00640 1 Processor -- start 2024-04-26T20:01:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.852+0000 7fd773c00640 1 -- start start 2024-04-26T20:01:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd773c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 0x7fd76c160480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd773c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd76c0cfc30 0x7fd76c0d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd773c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c15f710 con 0x7fd76c0d1f70 2024-04-26T20:01:21.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd773c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd76c0d4000 con 0x7fd76c0cfc30 2024-04-26T20:01:21.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd773c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c160c20 con 0x7fd76c15f330 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.853+0000 7fd771c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd76c0d1f70 0x7fd76c162080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36066/0 (socket says 172.21.15.88:36066) 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd771c00640 1 -- 172.21.15.88:0/2414533690 learned_addr learned my addr 172.21.15.88:0/2414533690 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76be00640 1 --2- 172.21.15.88:0/2414533690 >> v2:172.21.15.132:3300/0 conn(0x7fd76c0cfc30 0x7fd76c0d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd771200640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 0x7fd76c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620355415 0 0) 0x7fd76c15f710 con 0x7fd76c0d1f70 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd750003520 con 0x7fd76c0d1f70 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2463940178 0 0) 0x7fd750003520 con 0x7fd76c0d1f70 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 msgr2=0x7fd76c160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:21.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 0x7fd76c160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 >> v2:172.21.15.132:3300/0 conn(0x7fd76c0cfc30 msgr2=0x7fd76c0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 --2- 172.21.15.88:0/2414533690 >> v2:172.21.15.132:3300/0 conn(0x7fd76c0cfc30 0x7fd76c0d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c0d0510 con 0x7fd76c0d1f70 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd7640019f0 con 0x7fd76c0d1f70 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 >> v1:172.21.15.88:6789/0 conn(0x7fd76c0d1f70 legacy=0x7fd76c162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7500039b0 0x7fd750003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.854+0000 7fd76b400640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 0x7fd750004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd76b400640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd750004d40 0x7fd7500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0d4000 con 0x7fd7500039b0 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c160c20 con 0x7fd750004d40 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd75000b9c0 con 0x7fd76c15f330 2024-04-26T20:01:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7500039b0 0x7fd750003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 -- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 msgr2=0x7fd750004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd76c15f330 0x7fd750004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 -- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd750004d40 msgr2=0x7fd7500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd750004d40 0x7fd7500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.855+0000 7fd771200640 1 -- 172.21.15.88:0/2414533690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd750003520 con 0x7fd7500039b0 2024-04-26T20:01:21.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.856+0000 7fd771200640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7500039b0 0x7fd750003dd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009ac0 tx=0x7fd750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7229ad0753989804 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:21.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.857+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c036070 con 0x7fd7500039b0 2024-04-26T20:01:21.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.857+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c031400 con 0x7fd7500039b0 2024-04-26T20:01:21.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.857+0000 7fd76b400640 1 -- 172.21.15.88:0/2414533690 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c009690 con 0x7fd7500039b0 2024-04-26T20:01:21.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.857+0000 7fd773c00640 1 -- 172.21.15.88:0/2414533690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7500039b0 msgr2=0x7fd750003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.857+0000 7fd773c00640 1 --2- 172.21.15.88:0/2414533690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7500039b0 0x7fd750003dd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009ac0 tx=0x7fd750005830 comp rx=0 tx=0).stop 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.858+0000 7fd771c00640 1 -- 172.21.15.88:0/2414533690 reap_dead start 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.858+0000 7fd773c00640 1 -- 172.21.15.88:0/2414533690 shutdown_connections 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.858+0000 7fd773c00640 1 -- 172.21.15.88:0/2414533690 >> 172.21.15.88:0/2414533690 conn(0x7fd76c09e3b0 msgr2=0x7fd76c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.858+0000 7fd773c00640 1 -- 172.21.15.88:0/2414533690 shutdown_connections 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.858+0000 7fd773c00640 1 -- 172.21.15.88:0/2414533690 wait complete. 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.859+0000 7fd773c00640 1 Processor -- start 2024-04-26T20:01:21.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.860+0000 7fd773c00640 1 -- start start 2024-04-26T20:01:21.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.860+0000 7fd773c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7500039b0 0x7fd76c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd773c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd773c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 0x7fd76c0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd773c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0d0db0 con 0x7fd76c1f8490 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd773c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0d0ad0 con 0x7fd76c0c9d20 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd773c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c0d00b0 con 0x7fd7500039b0 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42664/0 (socket says 172.21.15.88:42664) 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 -- 172.21.15.88:0/3160138023 learned_addr learned my addr 172.21.15.88:0/3160138023 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd771200640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7500039b0 0x7fd76c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd771c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 0x7fd76c0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 -- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7500039b0 msgr2=0x7fd76c14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7500039b0 0x7fd76c14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 -- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 msgr2=0x7fd76c0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 0x7fd76c0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd76be00640 1 -- 172.21.15.88:0/3160138023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c0caae0 con 0x7fd76c1f8490 2024-04-26T20:01:21.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.861+0000 7fd771c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 0x7fd76c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd76be00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e860 tx=0x7fd75800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758017070 con 0x7fd76c1f8490 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75800c940 con 0x7fd76c1f8490 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75800cd10 con 0x7fd76c1f8490 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c0cadc0 con 0x7fd76c1f8490 2024-04-26T20:01:21.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.862+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c0cb380 con 0x7fd76c1f8490 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.863+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c069540 con 0x7fd76c1f8490 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.864+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd75801a070 con 0x7fd76c1f8490 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.865+0000 7fd768c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 0x7fd74c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.865+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd75808be80 con 0x7fd76c1f8490 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.865+0000 7fd771200640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 0x7fd74c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:21.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.866+0000 7fd771200640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 0x7fd74c06ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd75c033000 tx=0x7fd75c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:21.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:21.870+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd75805a210 con 0x7fd76c1f8490 2024-04-26T20:01:22.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.116+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd76c15eae0 con 0x7fd76c1f8490 2024-04-26T20:01:22.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.369+0000 7fd768c00640 1 -- 172.21.15.88:0/3160138023 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v16) ==== 136+0+0 (secure 0 0 0) 0x7fd758059b10 con 0x7fd76c1f8490 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 msgr2=0x7fd74c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 0x7fd74c06ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd75c033000 tx=0x7fd75c008030 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 msgr2=0x7fd76c1f8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75800e860 tx=0x7fd75800ed20 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 shutdown_connections 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd76c0c9d20 0x7fd76c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd76c1f8490 0x7fd76c1f8910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd74c06c920 0x7fd74c06ede0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 --2- 172.21.15.88:0/3160138023 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd7500039b0 0x7fd76c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.373+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 >> 172.21.15.88:0/3160138023 conn(0x7fd76c09e3b0 msgr2=0x7fd76c1615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:22.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.374+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 shutdown_connections 2024-04-26T20:01:22.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.374+0000 7fd773c00640 1 -- 172.21.15.88:0/3160138023 wait complete. 2024-04-26T20:01:22.396 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:01:22.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.594+0000 7f39fa600640 1 Processor -- start 2024-04-26T20:01:22.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.594+0000 7f39fa600640 1 -- start start 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39fa600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f39f406c770 0x7f39f406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39fa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 0x7f39f4073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39fa600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f406d090 con 0x7f39f415cb70 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39fa600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f39f40af460 con 0x7f39f406c770 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39fa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f39f40ae850 con 0x7f39f415cfd0 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39f2a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f39f415cb70 0x7f39f4073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36074/0 (socket says 172.21.15.88:36074) 2024-04-26T20:01:22.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.596+0000 7f39f2a00640 1 -- 172.21.15.88:0/2691959792 learned_addr learned my addr 172.21.15.88:0/2691959792 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f3400640 1 --2- 172.21.15.88:0/2691959792 >> v2:172.21.15.132:3300/0 conn(0x7f39f406c770 0x7f39f406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934835051 0 0) 0x7f39f406d090 con 0x7f39f415cb70 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39d8003520 con 0x7f39f415cb70 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f3e00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 0x7f39f4073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2031398484 0 0) 0x7f39d8003520 con 0x7f39f415cb70 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 msgr2=0x7f39f4073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:22.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 0x7f39f4073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 >> v2:172.21.15.132:3300/0 conn(0x7f39f406c770 msgr2=0x7f39f406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 --2- 172.21.15.88:0/2691959792 >> v2:172.21.15.132:3300/0 conn(0x7f39f406c770 0x7f39f406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f40aeb30 con 0x7f39f415cb70 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f39e0001670 con 0x7f39f415cb70 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 >> v1:172.21.15.88:6789/0 conn(0x7f39f415cb70 legacy=0x7f39f4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 0x7f39d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d8004d40 0x7f39d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39f40af460 con 0x7f39d8003f80 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f39f40ae850 con 0x7f39d8004d40 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.597+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f39d800dc80 con 0x7f39f415cfd0 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f3e00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d8004d40 0x7f39d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 -- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 msgr2=0x7f39d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:22.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f415cfd0 0x7f39d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 -- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d8004d40 msgr2=0x7f39d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:22.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39d8004d40 0x7f39d80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.598+0000 7f39f2a00640 1 -- 172.21.15.88:0/2691959792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d8003520 con 0x7f39d8003f80 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.599+0000 7f39f2a00640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39d8004800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c190 tx=0x7f39d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b2c7e6560fa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007c80 con 0x7f39d8003f80 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e0005ad0 con 0x7f39d8003f80 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39f2000640 1 -- 172.21.15.88:0/2691959792 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007070 con 0x7f39d8003f80 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 -- 172.21.15.88:0/2691959792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 msgr2=0x7f39d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 --2- 172.21.15.88:0/2691959792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39d8004800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c190 tx=0x7f39d8005830 comp rx=0 tx=0).stop 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39f3e00640 1 -- 172.21.15.88:0/2691959792 reap_dead start 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 -- 172.21.15.88:0/2691959792 shutdown_connections 2024-04-26T20:01:22.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 -- 172.21.15.88:0/2691959792 >> 172.21.15.88:0/2691959792 conn(0x7f39f40b2d50 msgr2=0x7f39f40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:22.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 -- 172.21.15.88:0/2691959792 shutdown_connections 2024-04-26T20:01:22.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.600+0000 7f39fa600640 1 -- 172.21.15.88:0/2691959792 wait complete. 2024-04-26T20:01:22.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.602+0000 7f39fa600640 1 Processor -- start 2024-04-26T20:01:22.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.602+0000 7f39fa600640 1 -- start start 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f4072820 0x7f39f415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f4072ca0 0x7f39f40cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39f415c600 con 0x7f39d8003f80 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f39f40af0f0 con 0x7f39f4072ca0 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.603+0000 7f39fa600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f39f415c780 con 0x7f39f4072820 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42690/0 (socket says 172.21.15.88:42690) 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 -- 172.21.15.88:0/1510580083 learned_addr learned my addr 172.21.15.88:0/1510580083 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3e00640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f4072ca0 0x7f39f40cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f2a00640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f4072820 0x7f39f415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 -- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f4072820 msgr2=0x7f39f415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f4072820 0x7f39f415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 -- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f4072ca0 msgr2=0x7f39f40cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f4072ca0 0x7f39f40cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 -- 172.21.15.88:0/1510580083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f415bb00 con 0x7f39d8003f80 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.604+0000 7f39f3400640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f39e400e860 tx=0x7f39e400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.605+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4017070 con 0x7f39d8003f80 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.605+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e400c940 con 0x7f39d8003f80 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.605+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e400cd10 con 0x7f39d8003f80 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.605+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f415b820 con 0x7f39d8003f80 2024-04-26T20:01:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.605+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f41f8a10 con 0x7f39d8003f80 2024-04-26T20:01:22.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.606+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f4069540 con 0x7f39d8003f80 2024-04-26T20:01:22.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.607+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39e401a070 con 0x7f39d8003f80 2024-04-26T20:01:22.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.608+0000 7f39eb400640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 0x7f39d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:22.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.608+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f39e408bc10 con 0x7f39d8003f80 2024-04-26T20:01:22.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.609+0000 7f39f2a00640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 0x7f39d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:22.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.609+0000 7f39f2a00640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 0x7f39d406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f39e0032000 tx=0x7f39e0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.613+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39e4059fa0 con 0x7f39d8003f80 2024-04-26T20:01:22.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.844+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f39f40cd430 con 0x7f39d406c8f0 2024-04-26T20:01:22.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.848+0000 7f39eb400640 1 -- 172.21.15.88:0/1510580083 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f39f40cd430 con 0x7f39d406c8f0 2024-04-26T20:01:22.849 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:01:22.849 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:22.849 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-04-26T20:01:22.850 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:22.850 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 msgr2=0x7f39d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 0x7f39d406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f39e0032000 tx=0x7f39e0009030 comp rx=0 tx=0).stop 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 msgr2=0x7f39f41f9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f39e400e860 tx=0x7f39e400ed20 comp rx=0 tx=0).stop 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 shutdown_connections 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39f4072ca0 0x7f39f40cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f39d406c8f0 0x7f39d406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f39f4072820 0x7f39f415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 --2- 172.21.15.88:0/1510580083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39d8003f80 0x7f39f41f9210 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 >> 172.21.15.88:0/1510580083 conn(0x7f39f40b2d50 msgr2=0x7f39f415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.850+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 shutdown_connections 2024-04-26T20:01:22.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:22.851+0000 7f39fa600640 1 -- 172.21.15.88:0/1510580083 wait complete. 2024-04-26T20:01:22.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:01:23.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.072+0000 7f6726000640 1 Processor -- start 2024-04-26T20:01:23.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.072+0000 7f6726000640 1 -- start start 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.073+0000 7f6726000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f672015e220 0x7f672006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.073+0000 7f6726000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672006cba0 con 0x7f6720071860 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f6726000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672015d740 con 0x7f672015d030 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f6726000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f672015dd90 con 0x7f672015e220 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f671fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f6720071860 0x7f67200aee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36086/0 (socket says 172.21.15.88:36086) 2024-04-26T20:01:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f671fe00640 1 -- 172.21.15.88:0/457123682 learned_addr learned my addr 172.21.15.88:0/457123682 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:23.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f671ea00640 1 --2- 172.21.15.88:0/457123682 >> v2:172.21.15.132:3300/0 conn(0x7f672015e220 0x7f672006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288751905 0 0) 0x7f672006cba0 con 0x7f6720071860 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.074+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6704003520 con 0x7f6720071860 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1926224537 0 0) 0x7f6704003520 con 0x7f6720071860 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 >> v2:172.21.15.132:3300/0 conn(0x7f672015e220 msgr2=0x7f672006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 --2- 172.21.15.88:0/457123682 >> v2:172.21.15.132:3300/0 conn(0x7f672015e220 0x7f672006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 >> v1:172.21.15.132:6789/0 conn(0x7f672015d030 legacy=0x7f672006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672015da20 con 0x7f6720071860 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f67180019f0 con 0x7f6720071860 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 >> v1:172.21.15.88:6789/0 conn(0x7f6720071860 legacy=0x7f67200aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040039b0 0x7f6704003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6704004800 0x7f6704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 0x7f670400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f672015dd90 con 0x7f67040051c0 2024-04-26T20:01:23.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f670400c090 con 0x7f67040039b0 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f670400c290 con 0x7f6704004800 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.075+0000 7f671fe00640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 0x7f670400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671f400640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040039b0 0x7f6704003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671fe00640 1 -- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6704004800 msgr2=0x7f6704004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671fe00640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6704004800 0x7f6704004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671fe00640 1 -- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040039b0 msgr2=0x7f6704003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671fe00640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040039b0 0x7f6704003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.076+0000 7f671fe00640 1 -- 172.21.15.88:0/457123682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6704003520 con 0x7f67040051c0 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.077+0000 7f671fe00640 1 -- 172.21.15.88:0/457123682 reap_dead start 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.077+0000 7f671fe00640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 0x7f670400b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f671800a840 tx=0x7f6704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7641707dad43ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.077+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800a460 con 0x7f67040051c0 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.077+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671802f070 con 0x7f67040051c0 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.077+0000 7f671e000640 1 -- 172.21.15.88:0/457123682 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671802bcb0 con 0x7f67040051c0 2024-04-26T20:01:23.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 -- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 msgr2=0x7f670400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 0x7f670400b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f671800a840 tx=0x7f6704005830 comp rx=0 tx=0).stop 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 -- 172.21.15.88:0/457123682 shutdown_connections 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 --2- 172.21.15.88:0/457123682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67040051c0 0x7f670400b9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 -- 172.21.15.88:0/457123682 >> 172.21.15.88:0/457123682 conn(0x7f67200d1e00 msgr2=0x7f67200d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 -- 172.21.15.88:0/457123682 shutdown_connections 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.078+0000 7f6726000640 1 -- 172.21.15.88:0/457123682 wait complete. 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.080+0000 7f6726000640 1 Processor -- start 2024-04-26T20:01:23.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.080+0000 7f6726000640 1 -- start start 2024-04-26T20:01:23.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 0x7f6720155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67201f8650 0x7f6720152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f672006c4d0 con 0x7f67201f81d0 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f672006c350 con 0x7f67040051c0 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f6726000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f672006c650 con 0x7f67201f8650 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f671ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f671ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42712/0 (socket says 172.21.15.88:42712) 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f671ea00640 1 -- 172.21.15.88:0/949435594 learned_addr learned my addr 172.21.15.88:0/949435594 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.081+0000 7f671f400640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 0x7f6720155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 -- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67201f8650 msgr2=0x7f6720152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67201f8650 0x7f6720152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 -- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 msgr2=0x7f6720155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 0x7f6720155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 -- 172.21.15.88:0/949435594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67200b5610 con 0x7f67201f81d0 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671f400640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 0x7f6720155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:23.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f671ea00640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f670800e7a0 tx=0x7f670800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f67201f81d0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670800c940 con 0x7f67201f81d0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800cd10 con 0x7f67201f81d0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67200b58f0 con 0x7f67201f81d0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.082+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67200b5eb0 con 0x7f67201f81d0 2024-04-26T20:01:23.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.084+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6720069650 con 0x7f67201f81d0 2024-04-26T20:01:23.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.085+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f67201f81d0 2024-04-26T20:01:23.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.086+0000 7f6717400640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 0x7f66fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.086+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f670808bd00 con 0x7f67201f81d0 2024-04-26T20:01:23.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.086+0000 7f671f400640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 0x7f66fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.087+0000 7f671f400640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 0x7f66fc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.090+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f670805a1c0 con 0x7f67201f81d0 2024-04-26T20:01:23.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.295+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f67200767d0 con 0x7f67201f81d0 2024-04-26T20:01:23.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.296+0000 7f6717400640 1 -- 172.21.15.88:0/949435594 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f6708059ac0 con 0x7f67201f81d0 2024-04-26T20:01:23.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 msgr2=0x7f66fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 0x7f66fc06ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710009040 comp rx=0 tx=0).stop 2024-04-26T20:01:23.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 msgr2=0x7f6720152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f670800e7a0 tx=0x7f670800ec60 comp rx=0 tx=0).stop 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 shutdown_connections 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f66fc06c920 0x7f66fc06ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67201f8650 0x7f6720152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67201f81d0 0x7f6720152740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 --2- 172.21.15.88:0/949435594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67040051c0 0x7f6720155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.299+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 >> 172.21.15.88:0/949435594 conn(0x7f67200d1e00 msgr2=0x7f67200b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.300+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 shutdown_connections 2024-04-26T20:01:23.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.300+0000 7f6726000640 1 -- 172.21.15.88:0/949435594 wait complete. 2024-04-26T20:01:23.320 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create vol27560 2024-04-26T20:01:23.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.518+0000 7faad6a00640 1 Processor -- start 2024-04-26T20:01:23.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.519+0000 7faad6a00640 1 -- start start 2024-04-26T20:01:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.520+0000 7faad6a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faad0071e80 0x7faad00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.520+0000 7faad6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad0072970 0x7faad00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.520+0000 7faad6a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad006e320 con 0x7faad00b4ed0 2024-04-26T20:01:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.520+0000 7faad6a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7faad006e600 con 0x7faad0071e80 2024-04-26T20:01:23.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.520+0000 7faad6a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faad006e8e0 con 0x7faad0072970 2024-04-26T20:01:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.521+0000 7faad5600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faad0071e80 0x7faad00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.521+0000 7faad6000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad0072970 0x7faad00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.521+0000 7faad5600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faad0071e80 0x7faad00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:39014/0 (socket says 172.21.15.88:39014) 2024-04-26T20:01:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.521+0000 7faad5600640 1 -- 172.21.15.88:0/2491264093 learned_addr learned my addr 172.21.15.88:0/2491264093 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:23.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.522+0000 7faad5600640 1 -- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad0072970 msgr2=0x7faad00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.522+0000 7faad5600640 1 --2- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad0072970 0x7faad00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.522+0000 7faad5600640 1 -- 172.21.15.88:0/2491264093 >> v1:172.21.15.132:6789/0 conn(0x7faad00b4ed0 legacy=0x7faad0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.522+0000 7faad5600640 1 -- 172.21.15.88:0/2491264093 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad006ebc0 con 0x7faad0071e80 2024-04-26T20:01:23.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.523+0000 7faad5600640 1 --2- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad00b3ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9eedee27b1e277a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faacfe00640 1 -- 172.21.15.88:0/2491264093 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4037070 con 0x7faad0071e80 2024-04-26T20:01:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faacfe00640 1 -- 172.21.15.88:0/2491264093 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac4032440 con 0x7faad0071e80 2024-04-26T20:01:23.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faacfe00640 1 -- 172.21.15.88:0/2491264093 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4032810 con 0x7faad0071e80 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 -- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 msgr2=0x7faad00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 --2- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad00b3ee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a250 comp rx=0 tx=0).stop 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 -- 172.21.15.88:0/2491264093 shutdown_connections 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 --2- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad0072970 0x7faad00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 --2- 172.21.15.88:0/2491264093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad00b3ee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 -- 172.21.15.88:0/2491264093 >> 172.21.15.88:0/2491264093 conn(0x7faad00b2900 msgr2=0x7faad00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:23.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 -- 172.21.15.88:0/2491264093 shutdown_connections 2024-04-26T20:01:23.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.524+0000 7faad6a00640 1 -- 172.21.15.88:0/2491264093 wait complete. 2024-04-26T20:01:23.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.526+0000 7faad6a00640 1 Processor -- start 2024-04-26T20:01:23.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.527+0000 7faad6a00640 1 -- start start 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 0x7faad01444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faad006f0e0 con 0x7faad0072970 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faad00b5ff0 con 0x7faad0071e80 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad6a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faad006f260 con 0x7faad00b4ed0 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42714/0 (socket says 172.21.15.88:42714) 2024-04-26T20:01:23.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.528+0000 7faad4c00640 1 -- 172.21.15.88:0/232205265 learned_addr learned my addr 172.21.15.88:0/232205265 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad5600640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad6000640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 0x7faad01444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 -- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 msgr2=0x7faad01444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 0x7faad01444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 -- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 msgr2=0x7faad014ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad014ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 -- 172.21.15.88:0/232205265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015f780 con 0x7faad0072970 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad6000640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 0x7faad01444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad4c00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c3f0 tx=0x7faabc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc0052d0 con 0x7faad0072970 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc005490 con 0x7faad0072970 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc007260 con 0x7faad0072970 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.529+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad015fa20 con 0x7faad0072970 2024-04-26T20:01:23.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.530+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad015ffe0 con 0x7faad0072970 2024-04-26T20:01:23.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.531+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad00693c0 con 0x7faad0072970 2024-04-26T20:01:23.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.534+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faabc007400 con 0x7faad0072970 2024-04-26T20:01:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.536+0000 7faacd600640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.536+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faabc08afe0 con 0x7faad0072970 2024-04-26T20:01:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.536+0000 7faad5600640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.537+0000 7faad5600640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 0x7faab006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:23.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.540+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faabc059440 con 0x7faad0072970 2024-04-26T20:01:23.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:23.766+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "vol27560", "target": ["mon-mgr", ""]}) -- 0x7faad00af0b0 con 0x7faab006c8f0 2024-04-26T20:01:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.440+0000 7faacd600640 1 -- 172.21.15.88:0/232205265 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7faad00af0b0 con 0x7faab006c8f0 2024-04-26T20:01:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:Volume created successfully (no MDS daemons created) 2024-04-26T20:01:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 msgr2=0x7faab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 0x7faab006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 msgr2=0x7faad0154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c3f0 tx=0x7faabc00c8b0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 shutdown_connections 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faab006c8f0 0x7faab006edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faad00b4ed0 0x7faad01444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad0072970 0x7faad0154c10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 --2- 172.21.15.88:0/232205265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faad0071e80 0x7faad014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 >> 172.21.15.88:0/232205265 conn(0x7faad00b2900 msgr2=0x7faad00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 shutdown_connections 2024-04-26T20:01:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.444+0000 7faad6a00640 1 -- 172.21.15.88:0/232205265 wait complete. 2024-04-26T20:01:26.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:01:26.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.647+0000 7f5b8c400640 1 Processor -- start 2024-04-26T20:01:26.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.647+0000 7f5b8c400640 1 -- start start 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.648+0000 7f5b8c400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5b8415cdf0 0x7f5b840add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8c400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 0x7f5b840b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8c400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b840af9a0 con 0x7f5b840cd650 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8c400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840ae2c0 con 0x7f5b8415cdf0 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8c400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840b05b0 con 0x7f5b8406ce80 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5b8415cdf0 0x7f5b840add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f5b8415cdf0 0x7f5b840add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42724/0 (socket says 172.21.15.88:42724) 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 -- 172.21.15.88:0/613034247 learned_addr learned my addr 172.21.15.88:0/613034247 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:26.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8a400640 1 --2- 172.21.15.88:0/613034247 >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 0x7f5b840b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 -- 172.21.15.88:0/613034247 >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 msgr2=0x7f5b840b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 --2- 172.21.15.88:0/613034247 >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 0x7f5b840b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 -- 172.21.15.88:0/613034247 >> v1:172.21.15.132:6789/0 conn(0x7f5b840cd650 legacy=0x7f5b8406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b89a00640 1 -- 172.21.15.88:0/613034247 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840afc80 con 0x7f5b8415cdf0 2024-04-26T20:01:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.649+0000 7f5b8a400640 1 --2- 172.21.15.88:0/613034247 >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 0x7f5b840b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.650+0000 7f5b89a00640 1 --2- 172.21.15.88:0/613034247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8415cdf0 0x7f5b840add80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d0380 tx=0x7f5b7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c231fa33b763f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b83e00640 1 -- 172.21.15.88:0/613034247 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78037070 con 0x7f5b8415cdf0 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b83e00640 1 -- 172.21.15.88:0/613034247 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78032440 con 0x7f5b8415cdf0 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b83e00640 1 -- 172.21.15.88:0/613034247 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78032810 con 0x7f5b8415cdf0 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 -- 172.21.15.88:0/613034247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8415cdf0 msgr2=0x7f5b840add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 --2- 172.21.15.88:0/613034247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8415cdf0 0x7f5b840add80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d0380 tx=0x7f5b7802a230 comp rx=0 tx=0).stop 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 -- 172.21.15.88:0/613034247 shutdown_connections 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 --2- 172.21.15.88:0/613034247 >> v2:172.21.15.132:3300/0 conn(0x7f5b8406ce80 0x7f5b840b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 --2- 172.21.15.88:0/613034247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8415cdf0 0x7f5b840add80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.651+0000 7f5b8c400640 1 -- 172.21.15.88:0/613034247 >> 172.21.15.88:0/613034247 conn(0x7f5b840b2d50 msgr2=0x7f5b840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.652+0000 7f5b8c400640 1 -- 172.21.15.88:0/613034247 shutdown_connections 2024-04-26T20:01:26.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.652+0000 7f5b8c400640 1 -- 172.21.15.88:0/613034247 wait complete. 2024-04-26T20:01:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.653+0000 7f5b8c400640 1 Processor -- start 2024-04-26T20:01:26.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.653+0000 7f5b8c400640 1 -- start start 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b840cd650 0x7f5b840d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b8415cdf0 0x7f5b8414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8415d1d0 con 0x7f5b8406ce80 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840b0240 con 0x7f5b840cd650 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.654+0000 7f5b8c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8415d350 con 0x7f5b8415cdf0 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42732/0 (socket says 172.21.15.88:42732) 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 -- 172.21.15.88:0/306123956 learned_addr learned my addr 172.21.15.88:0/306123956 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 -- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b8415cdf0 msgr2=0x7f5b8414cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89000640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b840cd650 0x7f5b840d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:26.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b8415cdf0 0x7f5b8414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 -- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b840cd650 msgr2=0x7f5b840d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b840cd650 0x7f5b840d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 -- 172.21.15.88:0/306123956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8414cc50 con 0x7f5b8406ce80 2024-04-26T20:01:26.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.655+0000 7f5b89a00640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000c00 tx=0x7f5b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.656+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78041020 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.656+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78030820 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.656+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78037040 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.656+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b840cf970 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.656+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840cff30 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.657+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b84069540 con 0x7f5b8406ce80 2024-04-26T20:01:26.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.658+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b78044020 con 0x7f5b8406ce80 2024-04-26T20:01:26.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.659+0000 7f5b81600640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.659+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6547+0+0 (secure 0 0 0) 0x7f5b780ac7f0 con 0x7f5b8406ce80 2024-04-26T20:01:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.659+0000 7f5b89000640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:26.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.660+0000 7f5b89000640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 0x7f5b6406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d1060 tx=0x7f5b70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:26.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.664+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b7804b030 con 0x7f5b8406ce80 2024-04-26T20:01:26.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.887+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5b8415c2d0 con 0x7f5b6406c8f0 2024-04-26T20:01:26.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.890+0000 7f5b81600640 1 -- 172.21.15.88:0/306123956 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+81 (secure 0 0 0) 0x7f5b8415c2d0 con 0x7f5b6406c8f0 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "vol27560" 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:26.892 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:01:26.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 msgr2=0x7f5b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 0x7f5b6406edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5b840d1060 tx=0x7f5b70009040 comp rx=0 tx=0).stop 2024-04-26T20:01:26.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 msgr2=0x7f5b8414c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000c00 tx=0x7f5b780046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 shutdown_connections 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5b8415cdf0 0x7f5b8414cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b840cd650 0x7f5b840d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 --2- 172.21.15.88:0/306123956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b8406ce80 0x7f5b8414c710 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.892+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 >> 172.21.15.88:0/306123956 conn(0x7f5b840b2d50 msgr2=0x7f5b8406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.893+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 shutdown_connections 2024-04-26T20:01:26.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:26.893+0000 7f5b8c400640 1 -- 172.21.15.88:0/306123956 wait complete. 2024-04-26T20:01:26.913 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-04-26T20:01:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.098+0000 7f0132000640 1 Processor -- start 2024-04-26T20:01:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.098+0000 7f0132000640 1 -- start start 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f0132000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f012c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f0132000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012c06eea0 con 0x7f012c0b3c30 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f0132000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012c06f140 con 0x7f012c156650 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f0132000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f012c06f420 con 0x7f012c06c770 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f012be00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f012c0b3c30 0x7f012c0b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36102/0 (socket says 172.21.15.88:36102) 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.100+0000 7f012be00640 1 -- 172.21.15.88:0/3143371410 learned_addr learned my addr 172.21.15.88:0/3143371410 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f0123e00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f012c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479305366 0 0) 0x7f012c06eea0 con 0x7f012c0b3c30 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0110005730 con 0x7f012c0b3c30 2024-04-26T20:01:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1778918918 0 0) 0x7f0110005730 con 0x7f012c0b3c30 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 msgr2=0x7f012c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f012c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 >> v1:172.21.15.132:6789/0 conn(0x7f012c156650 legacy=0x7f012c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c06f700 con 0x7f012c0b3c30 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f01240019f0 con 0x7f012c0b3c30 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 >> v1:172.21.15.88:6789/0 conn(0x7f012c0b3c30 legacy=0x7f012c0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.101+0000 7f012aa00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f0110005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012aa00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0110006170 0x7f0110006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012aa00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f01100073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f012c06f420 con 0x7f0110006f50 2024-04-26T20:01:27.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f011000dcb0 con 0x7f0110006170 2024-04-26T20:01:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f011000deb0 con 0x7f012c06c770 2024-04-26T20:01:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f01100073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012b400640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f0110005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 -- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 msgr2=0x7f0110005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c06c770 0x7f0110005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 -- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0110006170 msgr2=0x7f0110006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0110006170 0x7f0110006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.102+0000 7f012be00640 1 -- 172.21.15.88:0/3143371410 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0110005730 con 0x7f0110006f50 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.103+0000 7f012be00640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f01100073d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f012400a850 tx=0x7f0110007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109aadc173a0911e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400a4a0 con 0x7f0110006f50 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012402f070 con 0x7f0110006f50 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f012aa00640 1 -- 172.21.15.88:0/3143371410 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012402bd20 con 0x7f0110006f50 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 -- 172.21.15.88:0/3143371410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 msgr2=0x7f01100073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 --2- 172.21.15.88:0/3143371410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f01100073d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f012400a850 tx=0x7f0110007a40 comp rx=0 tx=0).stop 2024-04-26T20:01:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f012be00640 1 -- 172.21.15.88:0/3143371410 reap_dead start 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 -- 172.21.15.88:0/3143371410 shutdown_connections 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 -- 172.21.15.88:0/3143371410 >> 172.21.15.88:0/3143371410 conn(0x7f012c0b2d50 msgr2=0x7f012c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 -- 172.21.15.88:0/3143371410 shutdown_connections 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.104+0000 7f0132000640 1 -- 172.21.15.88:0/3143371410 wait complete. 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.106+0000 7f0132000640 1 Processor -- start 2024-04-26T20:01:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.106+0000 7f0132000640 1 -- start start 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f012c158e10 0x7f012c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c1eeda0 0x7f012c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f012c15c680 con 0x7f0110006f50 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f012c06fcc0 con 0x7f012c158e10 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f0132000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f012c1595b0 con 0x7f012c1eeda0 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f012b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f012b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42750/0 (socket says 172.21.15.88:42750) 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.107+0000 7f012b400640 1 -- 172.21.15.88:0/3740528500 learned_addr learned my addr 172.21.15.88:0/3740528500 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 -- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c1eeda0 msgr2=0x7f012c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c1eeda0 0x7f012c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 -- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f012c158e10 msgr2=0x7f012c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f012c158e10 0x7f012c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 -- 172.21.15.88:0/3740528500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c0720b0 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f012b400640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e550 tx=0x7f011c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c0052e0 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f011c018070 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c003810 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c072390 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.108+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c072950 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.109+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c069650 con 0x7f0110006f50 2024-04-26T20:01:27.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.111+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f011c0039b0 con 0x7f0110006f50 2024-04-26T20:01:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.112+0000 7f0123400640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 0x7f010806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.112+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6547+0+0 (secure 0 0 0) 0x7f011c08bb70 con 0x7f0110006f50 2024-04-26T20:01:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.112+0000 7f0123e00640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 0x7f010806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.113+0000 7f0123e00640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 0x7f010806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f012c15c0e0 tx=0x7f01140094e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.117+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f011c059c40 con 0x7f0110006f50 2024-04-26T20:01:27.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.302+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) -- 0x7f012c159840 con 0x7f0110006f50 2024-04-26T20:01:27.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.305+0000 7f0123400640 1 -- 172.21.15.88:0/3740528500 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v36) ==== 97+0+9302 (secure 0 0 0) 0x7f011c059540 con 0x7f0110006f50 2024-04-26T20:01:27.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 msgr2=0x7f010806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 0x7f010806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f012c15c0e0 tx=0x7f01140094e0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 msgr2=0x7f012c1f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f011c00e550 tx=0x7f011c00ea10 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 shutdown_connections 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f010806c8f0 0x7f010806edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f012c1eeda0 0x7f012c0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f012c158e10 0x7f012c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 --2- 172.21.15.88:0/3740528500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0110006f50 0x7f012c1f5a50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 >> 172.21.15.88:0/3740528500 conn(0x7f012c0b2d50 msgr2=0x7f012c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 shutdown_connections 2024-04-26T20:01:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.308+0000 7f0132000640 1 -- 172.21.15.88:0/3740528500 wait complete. 2024-04-26T20:01:27.327 INFO:teuthology.orchestra.run.smithi088.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":6,"pool_name":"cephfs.vol27560.meta","create_time":"2024-04-26T20:01:23.780869+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"vol27560"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool_id":7,"pool_name":"cephfs.vol27560.data","create_time":"2024-04-26T20:01:24.436359+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"vol27560"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}] 2024-04-26T20:01:27.327 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get vol27560 --format=json 2024-04-26T20:01:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.532+0000 7f33cf600640 1 Processor -- start 2024-04-26T20:01:27.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.532+0000 7f33cf600640 1 -- start start 2024-04-26T20:01:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33cf600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33c815d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33cf600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c815dca0 con 0x7f33c80b5e30 2024-04-26T20:01:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33cf600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c815df80 con 0x7f33c809c160 2024-04-26T20:01:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33cf600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f33c8154520 con 0x7f33c80b73a0 2024-04-26T20:01:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33c7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f33c80b5e30 0x7f33c809ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36108/0 (socket says 172.21.15.88:36108) 2024-04-26T20:01:27.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33c7e00640 1 -- 172.21.15.88:0/4105718878 learned_addr learned my addr 172.21.15.88:0/4105718878 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138114399 0 0) 0x7f33c815dca0 con 0x7f33c80b5e30 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.534+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33ac003520 con 0x7f33c80b5e30 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33ccc00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33c815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2816155050 0 0) 0x7f33ac003520 con 0x7f33c80b5e30 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 msgr2=0x7f33c815d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33c815d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 >> v1:172.21.15.132:6789/0 conn(0x7f33c809c160 legacy=0x7f33c8070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c815e260 con 0x7f33c80b5e30 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f33b0001670 con 0x7f33c80b5e30 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 >> v1:172.21.15.88:6789/0 conn(0x7f33c80b5e30 legacy=0x7f33c809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ac004d40 0x7f33ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8154520 con 0x7f33ac004010 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac00bb80 con 0x7f33ac004d40 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac00bd80 con 0x7f33c80b73a0 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.535+0000 7f33c7e00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33c7e00640 1 -- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 msgr2=0x7f33ac0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33ccc00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33c7e00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c80b73a0 0x7f33ac0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33cd600640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ac004d40 0x7f33ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33c7e00640 1 -- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ac004d40 msgr2=0x7f33ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33c7e00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ac004d40 0x7f33ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33c7e00640 1 -- 172.21.15.88:0/4105718878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac003520 con 0x7f33ac004010 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.536+0000 7f33cd600640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ac004d40 0x7f33ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.537+0000 7f33c7e00640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33ac004800 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c240 tx=0x7f33ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421ceebe6a898755 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.537+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0007d10 con 0x7f33ac004010 2024-04-26T20:01:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.537+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b0005ad0 con 0x7f33ac004010 2024-04-26T20:01:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.537+0000 7f33c7400640 1 -- 172.21.15.88:0/4105718878 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0007070 con 0x7f33ac004010 2024-04-26T20:01:27.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 -- 172.21.15.88:0/4105718878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 msgr2=0x7f33ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 --2- 172.21.15.88:0/4105718878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33ac004800 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c240 tx=0x7f33ac005830 comp rx=0 tx=0).stop 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cd600640 1 -- 172.21.15.88:0/4105718878 reap_dead start 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 -- 172.21.15.88:0/4105718878 shutdown_connections 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 -- 172.21.15.88:0/4105718878 >> 172.21.15.88:0/4105718878 conn(0x7f33c8075f30 msgr2=0x7f33c80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 -- 172.21.15.88:0/4105718878 shutdown_connections 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.538+0000 7f33cf600640 1 -- 172.21.15.88:0/4105718878 wait complete. 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.539+0000 7f33cf600640 1 Processor -- start 2024-04-26T20:01:27.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 -- start start 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 0x7f33c8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 0x7f33c8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33c815d650 con 0x7f33ac004010 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f33c80b5970 con 0x7f33c809f180 2024-04-26T20:01:27.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.540+0000 7f33cf600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f33c815d7d0 con 0x7f33c81517b0 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42768/0 (socket says 172.21.15.88:42768) 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 -- 172.21.15.88:0/1500608398 learned_addr learned my addr 172.21.15.88:0/1500608398 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33c7e00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 0x7f33c8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33cd600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 0x7f33c8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 -- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 msgr2=0x7f33c8151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 0x7f33c8151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 -- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 msgr2=0x7f33c8151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 0x7f33c8151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33ccc00640 1 -- 172.21.15.88:0/1500608398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c809fa30 con 0x7f33ac004010 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33cd600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 0x7f33c8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.541+0000 7f33c7e00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 0x7f33c8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:27.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33ccc00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f33b8007e90 tx=0x7f33b800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8018020 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b80145f0 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c809fd10 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80149c0 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.542+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c80a02d0 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.543+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8069540 con 0x7f33ac004010 2024-04-26T20:01:27.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.544+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33b801b020 con 0x7f33ac004010 2024-04-26T20:01:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.546+0000 7f33c4c00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 0x7f33a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.546+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6547+0+0 (secure 0 0 0) 0x7f33b808cb00 con 0x7f33ac004010 2024-04-26T20:01:27.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.546+0000 7f33c7e00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 0x7f33a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:27.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.547+0000 7f33c7e00640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 0x7f33a406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:27.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.550+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33b8022020 con 0x7f33ac004010 2024-04-26T20:01:27.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.788+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "vol27560", "format": "json"} v 0) -- 0x7f33c809c990 con 0x7f33ac004010 2024-04-26T20:01:27.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.790+0000 7f33c4c00640 1 -- 172.21.15.88:0/1500608398 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "vol27560", "format": "json"}]=0 v18) ==== 95+0+2141 (secure 0 0 0) 0x7f33b805abe0 con 0x7f33ac004010 2024-04-26T20:01:27.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:27.792 INFO:teuthology.orchestra.run.smithi088.stdout:{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:25.425660+0000","modified":"2024-04-26T20:01:26.833950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4705":{"gid":4705,"name":"c","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.88:6837/2458488289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":2458488289},{"type":"v1","addr":"172.21.15.88:6837","nonce":2458488289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"vol27560","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4705,"qdb_cluster":[4705]},"id":3} 2024-04-26T20:01:27.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 msgr2=0x7f33a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 0x7f33a406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b002e000 comp rx=0 tx=0).stop 2024-04-26T20:01:27.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 msgr2=0x7f33c809ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:27.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f33b8007e90 tx=0x7f33b800f520 comp rx=0 tx=0).stop 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 shutdown_connections 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33a406c8f0 0x7f33a406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33c81517b0 0x7f33c8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33c809f180 0x7f33c8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.792+0000 7f33cf600640 1 --2- 172.21.15.88:0/1500608398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ac004010 0x7f33c809ec40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.793+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 >> 172.21.15.88:0/1500608398 conn(0x7f33c8075f30 msgr2=0x7f33c80d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.793+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 shutdown_connections 2024-04-26T20:01:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:27.793+0000 7f33cf600640 1 -- 172.21.15.88:0/1500608398 wait complete. 2024-04-26T20:01:27.804 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-04-26T20:01:27.802+0000 7fe577800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:01:27.814 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm vol27560 --yes-i-really-mean-it 2024-04-26T20:01:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.007+0000 7f6aacc00640 1 Processor -- start 2024-04-26T20:01:28.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.008+0000 7f6aacc00640 1 -- start start 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aacc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6aa80b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aacc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f6aa80b5ca0 0x7f6aa80b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aacc00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa80af800 con 0x7f6aa815d100 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aacc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa80ae340 con 0x7f6aa80b5ca0 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aacc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa80afaa0 con 0x7f6aa806ce80 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aa5e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f6aa815d100 0x7f6aa806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36118/0 (socket says 172.21.15.88:36118) 2024-04-26T20:01:28.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.009+0000 7f6aa5e00640 1 -- 172.21.15.88:0/3313350204 learned_addr learned my addr 172.21.15.88:0/3313350204 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267827812 0 0) 0x7f6aa80af800 con 0x7f6aa815d100 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3313350204 >> v2:172.21.15.132:3300/0 conn(0x7f6aa80b5ca0 0x7f6aa80b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a8c003520 con 0x7f6aa815d100 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa5400640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6aa80b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320069005 0 0) 0x7f6a8c003520 con 0x7f6aa815d100 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 msgr2=0x7f6aa80b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6aa80b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 >> v2:172.21.15.132:3300/0 conn(0x7f6aa80b5ca0 msgr2=0x7f6aa80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 --2- 172.21.15.88:0/3313350204 >> v2:172.21.15.132:3300/0 conn(0x7f6aa80b5ca0 0x7f6aa80b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa80afd80 con 0x7f6aa815d100 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f6a98001670 con 0x7f6aa815d100 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 >> v1:172.21.15.88:6789/0 conn(0x7f6aa815d100 legacy=0x7f6aa806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6a8c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a8c003f30 0x7f6a8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a8c004d40 0x7f6a8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa80ae340 con 0x7f6a8c004d40 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa80afaa0 con 0x7f6a8c003f30 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.010+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8c00b9c0 con 0x7f6aa806ce80 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a8c004d40 0x7f6a8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa5e00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6a8c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 -- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 msgr2=0x7f6a8c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa806ce80 0x7f6a8c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 -- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a8c003f30 msgr2=0x7f6a8c004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a8c003f30 0x7f6a8c004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.011+0000 7f6aa6800640 1 -- 172.21.15.88:0/3313350204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c003520 con 0x7f6a8c004d40 2024-04-26T20:01:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.012+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a8c004d40 0x7f6a8c0051c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fc0 tx=0x7f6a8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750070c68bd7ed23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.012+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0009c10 con 0x7f6a8c004d40 2024-04-26T20:01:28.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.012+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa0036070 con 0x7f6a8c004d40 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.012+0000 7f6aa4a00640 1 -- 172.21.15.88:0/3313350204 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa002d070 con 0x7f6a8c004d40 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 -- 172.21.15.88:0/3313350204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a8c004d40 msgr2=0x7f6a8c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3313350204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a8c004d40 0x7f6a8c0051c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009fc0 tx=0x7f6a8c005830 comp rx=0 tx=0).stop 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aa6800640 1 -- 172.21.15.88:0/3313350204 reap_dead start 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 -- 172.21.15.88:0/3313350204 shutdown_connections 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 -- 172.21.15.88:0/3313350204 >> 172.21.15.88:0/3313350204 conn(0x7f6aa80b2d50 msgr2=0x7f6aa80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 -- 172.21.15.88:0/3313350204 shutdown_connections 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.013+0000 7f6aacc00640 1 -- 172.21.15.88:0/3313350204 wait complete. 2024-04-26T20:01:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.015+0000 7f6aacc00640 1 Processor -- start 2024-04-26T20:01:28.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.015+0000 7f6aacc00640 1 -- start start 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa814b450 0x7f6aa80cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa814b830 0x7f6aa80b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa809ee50 con 0x7f6aa80b4970 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa80b0340 con 0x7f6aa814b830 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aacc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa8160b70 con 0x7f6aa814b450 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aa6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aa6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42798/0 (socket says 172.21.15.88:42798) 2024-04-26T20:01:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.016+0000 7f6aa6800640 1 -- 172.21.15.88:0/3994157594 learned_addr learned my addr 172.21.15.88:0/3994157594 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa5400640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa814b830 0x7f6aa80b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa5e00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa814b450 0x7f6aa80cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 -- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa814b450 msgr2=0x7f6aa80cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa814b450 0x7f6aa80cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 -- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa814b830 msgr2=0x7f6aa80b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa814b830 0x7f6aa80b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 -- 172.21.15.88:0/3994157594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa809df50 con 0x7f6aa80b4970 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6aa6800640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0033040 tx=0x7f6aa0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0009c10 con 0x7f6aa80b4970 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa0003bb0 con 0x7f6aa80b4970 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.017+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0004020 con 0x7f6aa80b4970 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.018+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa809e230 con 0x7f6aa80b4970 2024-04-26T20:01:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.018+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa809e7f0 con 0x7f6aa80b4970 2024-04-26T20:01:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.019+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa8069650 con 0x7f6aa80b4970 2024-04-26T20:01:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.020+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6aa0005d10 con 0x7f6aa80b4970 2024-04-26T20:01:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.021+0000 7f6a9e000640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.021+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6547+0+0 (secure 0 0 0) 0x7f6aa00ad3b0 con 0x7f6aa80b4970 2024-04-26T20:01:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.021+0000 7f6aa5e00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:28.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.022+0000 7f6aa5e00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 0x7f6a8806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a98001eb0 tx=0x7f6a980047e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:28.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.025+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6aa004a060 con 0x7f6aa80b4970 2024-04-26T20:01:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:28.250+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "vol27560", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f6aa8072d90 con 0x7f6a8806c8f0 2024-04-26T20:01:28.259 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-04-26T20:01:28.258+0000 7fad57a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:01:28.279 INFO:tasks.ceph.mds.c.smithi088.stderr: -13> 2024-04-26T20:01:27.802+0000 7fe577800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:01:28.327 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-04-26T20:01:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.291+0000 7f6a9e000640 1 -- 172.21.15.88:0/3994157594 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+80 (secure 0 0 0) 0x7f6aa8072d90 con 0x7f6a8806c8f0 2024-04-26T20:01:31.293 INFO:teuthology.orchestra.run.smithi088.stdout:metadata pool: cephfs.vol27560.meta data pool: ['cephfs.vol27560.data'] removed 2024-04-26T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 msgr2=0x7f6a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 0x7f6a8806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a98001eb0 tx=0x7f6a980047e0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 msgr2=0x7f6aa806e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0033040 tx=0x7f6aa0009750 comp rx=0 tx=0).stop 2024-04-26T20:01:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 shutdown_connections 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6aa80b4970 0x7f6aa806e010 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6aa814b830 0x7f6aa80b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 --2- 172.21.15.88:0/3994157594 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6aa814b450 0x7f6aa80cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 >> 172.21.15.88:0/3994157594 conn(0x7f6aa80b2d50 msgr2=0x7f6aa80b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:31.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.295+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 shutdown_connections 2024-04-26T20:01:31.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.296+0000 7f6aacc00640 1 -- 172.21.15.88:0/3994157594 wait complete. 2024-04-26T20:01:31.319 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:01:31.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:01:31.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:01:31.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:01:31.519 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:01:31.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:01:31.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:31.564 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:01:31.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:31.654 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.899+0000 7fdd74200640 1 Processor -- start 2024-04-26T20:01:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.899+0000 7fdd74200640 1 -- start start 2024-04-26T20:01:31.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd74200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdd6c15cdf0 0x7fdd6c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd74200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6c0af9a0 con 0x7fdd6c06ce80 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd74200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6c0afc80 con 0x7fdd6c0cd650 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd74200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6c0ae2c0 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd71800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdd6c15cdf0 0x7fdd6c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd72200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdd6c06ce80 0x7fdd6c0b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55530/0 (socket says 172.21.15.88:55530) 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.901+0000 7fdd72200640 1 -- 172.21.15.88:0/3343256439 learned_addr learned my addr 172.21.15.88:0/3343256439 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:31.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.902+0000 7fdd6be00640 1 -- 172.21.15.88:0/3343256439 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169728767 0 0) 0x7fdd6c0af9a0 con 0x7fdd6c06ce80 2024-04-26T20:01:31.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.902+0000 7fdd71800640 1 -- 172.21.15.88:0/3343256439 >> v1:172.21.15.132:6789/0 conn(0x7fdd6c0cd650 legacy=0x7fdd6c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:31.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.902+0000 7fdd71800640 1 -- 172.21.15.88:0/3343256439 >> v1:172.21.15.88:6789/0 conn(0x7fdd6c06ce80 legacy=0x7fdd6c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.902+0000 7fdd71800640 1 -- 172.21.15.88:0/3343256439 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c0aff60 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.903+0000 7fdd71800640 1 --2- 172.21.15.88:0/3343256439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cdf0 0x7fdd6c0add80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097a0 tx=0x7fdd6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8a9333c762ed82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:31.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.903+0000 7fdd6be00640 1 -- 172.21.15.88:0/3343256439 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60037070 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.903+0000 7fdd6be00640 1 -- 172.21.15.88:0/3343256439 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd60032440 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.903+0000 7fdd6be00640 1 -- 172.21.15.88:0/3343256439 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60032810 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 -- 172.21.15.88:0/3343256439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cdf0 msgr2=0x7fdd6c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 --2- 172.21.15.88:0/3343256439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cdf0 0x7fdd6c0add80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdd600097a0 tx=0x7fdd6002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 -- 172.21.15.88:0/3343256439 shutdown_connections 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 --2- 172.21.15.88:0/3343256439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cdf0 0x7fdd6c0add80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 -- 172.21.15.88:0/3343256439 >> 172.21.15.88:0/3343256439 conn(0x7fdd6c0b2d50 msgr2=0x7fdd6c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 -- 172.21.15.88:0/3343256439 shutdown_connections 2024-04-26T20:01:31.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.904+0000 7fdd74200640 1 -- 172.21.15.88:0/3343256439 wait complete. 2024-04-26T20:01:31.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.906+0000 7fdd74200640 1 Processor -- start 2024-04-26T20:01:31.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.906+0000 7fdd74200640 1 -- start start 2024-04-26T20:01:31.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 0x7fdd6c1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cdf0 0x7fdd6c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c15d1d0 con 0x7fdd6c0cd650 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c0b05b0 con 0x7fdd6c06ce80 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd74200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6c15d350 con 0x7fdd6c15cdf0 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57936/0 (socket says 172.21.15.88:57936) 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 -- 172.21.15.88:0/1913807546 learned_addr learned my addr 172.21.15.88:0/1913807546 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd71800640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 0x7fdd6c1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 -- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cdf0 msgr2=0x7fdd6c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cdf0 0x7fdd6c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 -- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 msgr2=0x7fdd6c1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 0x7fdd6c1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd70e00640 1 -- 172.21.15.88:0/1913807546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1e5b70 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.907+0000 7fdd71800640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 0x7fdd6c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd70e00640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c430 tx=0x7fdd5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58011790 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd58017070 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58011bb0 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1e5e50 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.908+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1e6410 con 0x7fdd6c0cd650 2024-04-26T20:01:31.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.909+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c069540 con 0x7fdd6c0cd650 2024-04-26T20:01:31.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.910+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd5801c460 con 0x7fdd6c0cd650 2024-04-26T20:01:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.911+0000 7fdd69600640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 0x7fdd4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.911+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fdd5808baf0 con 0x7fdd6c0cd650 2024-04-26T20:01:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.911+0000 7fdd71800640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 0x7fdd4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.912+0000 7fdd71800640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 0x7fdd4c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd60034000 tx=0x7fdd6002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:31.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:31.916+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd58059e20 con 0x7fdd6c0cd650 2024-04-26T20:01:32.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.116+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd6c069d20 con 0x7fdd6c0cd650 2024-04-26T20:01:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.118+0000 7fdd69600640 1 -- 172.21.15.88:0/1913807546 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16786 (secure 0 0 0) 0x7fdd58059720 con 0x7fdd6c0cd650 2024-04-26T20:01:32.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:32.120 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:31.270527+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:32.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 msgr2=0x7fdd4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 0x7fdd4c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd60034000 tx=0x7fdd6002e000 comp rx=0 tx=0).stop 2024-04-26T20:01:32.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 msgr2=0x7fdd6c0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c430 tx=0x7fdd5800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 shutdown_connections 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4c06c920 0x7fdd4c06ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cdf0 0x7fdd6c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c0cd650 0x7fdd6c0cf200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 --2- 172.21.15.88:0/1913807546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06ce80 0x7fdd6c1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 >> 172.21.15.88:0/1913807546 conn(0x7fdd6c0b2d50 msgr2=0x7fdd6c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.121+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 shutdown_connections 2024-04-26T20:01:32.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.122+0000 7fdd74200640 1 -- 172.21.15.88:0/1913807546 wait complete. 2024-04-26T20:01:32.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:03.091802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:01:04.331500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:32.142 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:32.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.339+0000 7fc724200640 1 Processor -- start 2024-04-26T20:01:32.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.339+0000 7fc724200640 1 -- start start 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc724200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc71c06ce80 0x7fc71c0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc724200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c0ade20 con 0x7fc71c147f20 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc724200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c0ae100 con 0x7fc71c073da0 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc724200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c0ae3e0 con 0x7fc71c06ce80 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc720e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc71c06ce80 0x7fc71c0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc720e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc71c06ce80 0x7fc71c0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57950/0 (socket says 172.21.15.88:57950) 2024-04-26T20:01:32.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.341+0000 7fc720e00640 1 -- 172.21.15.88:0/3138383039 learned_addr learned my addr 172.21.15.88:0/3138383039 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:32.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.342+0000 7fc720e00640 1 -- 172.21.15.88:0/3138383039 >> v1:172.21.15.132:6789/0 conn(0x7fc71c073da0 legacy=0x7fc71c0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:32.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.342+0000 7fc720e00640 1 -- 172.21.15.88:0/3138383039 >> v1:172.21.15.88:6789/0 conn(0x7fc71c147f20 legacy=0x7fc71c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.342+0000 7fc720e00640 1 -- 172.21.15.88:0/3138383039 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0ae6c0 con 0x7fc71c06ce80 2024-04-26T20:01:32.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.343+0000 7fc720e00640 1 --2- 172.21.15.88:0/3138383039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c06ce80 0x7fc71c0b5c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc708009870 tx=0x7fc70802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df317b01d6430a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.343+0000 7fc71be00640 1 -- 172.21.15.88:0/3138383039 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708037070 con 0x7fc71c06ce80 2024-04-26T20:01:32.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.343+0000 7fc71be00640 1 -- 172.21.15.88:0/3138383039 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc708032440 con 0x7fc71c06ce80 2024-04-26T20:01:32.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.343+0000 7fc71be00640 1 -- 172.21.15.88:0/3138383039 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708032810 con 0x7fc71c06ce80 2024-04-26T20:01:32.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 -- 172.21.15.88:0/3138383039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c06ce80 msgr2=0x7fc71c0b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 --2- 172.21.15.88:0/3138383039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c06ce80 0x7fc71c0b5c20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc708009870 tx=0x7fc70802a2a0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 -- 172.21.15.88:0/3138383039 shutdown_connections 2024-04-26T20:01:32.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 --2- 172.21.15.88:0/3138383039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c06ce80 0x7fc71c0b5c20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 -- 172.21.15.88:0/3138383039 >> 172.21.15.88:0/3138383039 conn(0x7fc71c0b2d50 msgr2=0x7fc71c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:32.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 -- 172.21.15.88:0/3138383039 shutdown_connections 2024-04-26T20:01:32.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.344+0000 7fc724200640 1 -- 172.21.15.88:0/3138383039 wait complete. 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.346+0000 7fc724200640 1 Processor -- start 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.346+0000 7fc724200640 1 -- start start 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc71c06ce80 0x7fc71c1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc71c147f20 0x7fc71c143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c06e220 con 0x7fc71c073da0 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c0741d0 con 0x7fc71c06ce80 2024-04-26T20:01:32.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc724200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c0af5d0 con 0x7fc71c147f20 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57956/0 (socket says 172.21.15.88:57956) 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 -- 172.21.15.88:0/1094611955 learned_addr learned my addr 172.21.15.88:0/1094611955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc721800640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc71c06ce80 0x7fc71c1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 -- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc71c147f20 msgr2=0x7fc71c143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc71c147f20 0x7fc71c143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 -- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc71c06ce80 msgr2=0x7fc71c1f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc71c06ce80 0x7fc71c1f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.347+0000 7fc720e00640 1 -- 172.21.15.88:0/1094611955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0cfb20 con 0x7fc71c073da0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc720e00640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc708009870 tx=0x7fc7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708041070 con 0x7fc71c073da0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc70803c870 con 0x7fc71c073da0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708037040 con 0x7fc71c073da0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c0cfe00 con 0x7fc71c073da0 2024-04-26T20:01:32.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.348+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c0d03c0 con 0x7fc71c073da0 2024-04-26T20:01:32.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.349+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c069650 con 0x7fc71c073da0 2024-04-26T20:01:32.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.350+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc708044070 con 0x7fc71c073da0 2024-04-26T20:01:32.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.352+0000 7fc719600640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.352+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fc7080ac900 con 0x7fc71c073da0 2024-04-26T20:01:32.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.352+0000 7fc721800640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.353+0000 7fc721800640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc710001500 tx=0x7fc710009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.356+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc70807ade0 con 0x7fc71c073da0 2024-04-26T20:01:32.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.557+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc71c0b6050 con 0x7fc71c073da0 2024-04-26T20:01:32.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.558+0000 7fc719600640 1 -- 172.21.15.88:0/1094611955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fc70807a6e0 con 0x7fc71c073da0 2024-04-26T20:01:32.560 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 msgr2=0x7fc6fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc710001500 tx=0x7fc710009040 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 msgr2=0x7fc71c143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc708009870 tx=0x7fc7080046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 shutdown_connections 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc6fc06c8f0 0x7fc6fc06edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc71c147f20 0x7fc71c143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc71c073da0 0x7fc71c143aa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 --2- 172.21.15.88:0/1094611955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc71c06ce80 0x7fc71c1f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 >> 172.21.15.88:0/1094611955 conn(0x7fc71c0b2d50 msgr2=0x7fc71c0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 shutdown_connections 2024-04-26T20:01:32.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.561+0000 7fc724200640 1 -- 172.21.15.88:0/1094611955 wait complete. 2024-04-26T20:01:32.582 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:01:32.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.782+0000 7fc557200640 1 Processor -- start 2024-04-26T20:01:32.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.783+0000 7fc557200640 1 -- start start 2024-04-26T20:01:32.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.784+0000 7fc557200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.784+0000 7fc557200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5500d58b0 0x7fc5500d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.784+0000 7fc557200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5500d40a0 con 0x7fc5500d1d10 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.784+0000 7fc557200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc5500b2070 con 0x7fc5500d58b0 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.784+0000 7fc557200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc5500d0960 con 0x7fc5500d3780 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5500d58b0 0x7fc5500d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc5500d58b0 0x7fc5500d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57964/0 (socket says 172.21.15.88:57964) 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 -- 172.21.15.88:0/2447551837 learned_addr learned my addr 172.21.15.88:0/2447551837 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54fe00640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 -- 172.21.15.88:0/2447551837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 msgr2=0x7fc5500d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 -- 172.21.15.88:0/2447551837 >> v1:172.21.15.88:6789/0 conn(0x7fc5500d1d10 legacy=0x7fc5500b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.785+0000 7fc54f400640 1 -- 172.21.15.88:0/2447551837 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5500d0c40 con 0x7fc5500d58b0 2024-04-26T20:01:32.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.786+0000 7fc54f400640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc5500d17d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc538009e50 tx=0x7fc53802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de894e9496f3a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.786+0000 7fc54ea00640 1 -- 172.21.15.88:0/2447551837 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538036070 con 0x7fc5500d58b0 2024-04-26T20:01:32.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.786+0000 7fc54ea00640 1 -- 172.21.15.88:0/2447551837 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538031600 con 0x7fc5500d58b0 2024-04-26T20:01:32.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc54ea00640 1 -- 172.21.15.88:0/2447551837 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380319d0 con 0x7fc5500d58b0 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 -- 172.21.15.88:0/2447551837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 msgr2=0x7fc5500d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc5500d17d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc538009e50 tx=0x7fc53802a710 comp rx=0 tx=0).stop 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 -- 172.21.15.88:0/2447551837 shutdown_connections 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc5500d17d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 --2- 172.21.15.88:0/2447551837 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 -- 172.21.15.88:0/2447551837 >> 172.21.15.88:0/2447551837 conn(0x7fc55009e4e0 msgr2=0x7fc5500d51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 -- 172.21.15.88:0/2447551837 shutdown_connections 2024-04-26T20:01:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.787+0000 7fc557200640 1 -- 172.21.15.88:0/2447551837 wait complete. 2024-04-26T20:01:32.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.789+0000 7fc557200640 1 Processor -- start 2024-04-26T20:01:32.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.789+0000 7fc557200640 1 -- start start 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 0x7fc55006dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500d32c0 con 0x7fc5500d58b0 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc5500d3140 con 0x7fc5500d1d10 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc557200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc55015f870 con 0x7fc5500d3780 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc555200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc555200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57966/0 (socket says 172.21.15.88:57966) 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.790+0000 7fc555200640 1 -- 172.21.15.88:0/1829223387 learned_addr learned my addr 172.21.15.88:0/1829223387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc54fe00640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 0x7fc55006dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 -- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 msgr2=0x7fc5500b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 -- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 msgr2=0x7fc55006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 0x7fc55006dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 -- 172.21.15.88:0/1829223387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5500b9c30 con 0x7fc5500d58b0 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc54fe00640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 0x7fc55006dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc555200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c3c0 tx=0x7fc54800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548017070 con 0x7fc5500d58b0 2024-04-26T20:01:32.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5480052d0 con 0x7fc5500d58b0 2024-04-26T20:01:32.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548003830 con 0x7fc5500d58b0 2024-04-26T20:01:32.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.791+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5500b9eb0 con 0x7fc5500d58b0 2024-04-26T20:01:32.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.792+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5500ba470 con 0x7fc5500d58b0 2024-04-26T20:01:32.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.793+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc550069650 con 0x7fc5500d58b0 2024-04-26T20:01:32.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.794+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc54801a070 con 0x7fc5500d58b0 2024-04-26T20:01:32.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.795+0000 7fc547e00640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 0x7fc52c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:32.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.795+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fc54808ae50 con 0x7fc5500d58b0 2024-04-26T20:01:32.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.795+0000 7fc54fe00640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 0x7fc52c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:32.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.796+0000 7fc54fe00640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 0x7fc52c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:32.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.799+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5500b9c30 con 0x7fc5500d58b0 2024-04-26T20:01:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.998+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5500d60b0 con 0x7fc5500d58b0 2024-04-26T20:01:33.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:32.999+0000 7fc547e00640 1 -- 172.21.15.88:0/1829223387 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7fc5480592b0 con 0x7fc5500d58b0 2024-04-26T20:01:33.001 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:01:33.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 msgr2=0x7fc52c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 0x7fc52c06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540008040 comp rx=0 tx=0).stop 2024-04-26T20:01:33.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 msgr2=0x7fc55014c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c3c0 tx=0x7fc54800c880 comp rx=0 tx=0).stop 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 shutdown_connections 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5500d58b0 0x7fc55014c3a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc52c06c8f0 0x7fc52c06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc5500d3780 0x7fc5500b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 --2- 172.21.15.88:0/1829223387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5500d1d10 0x7fc55006dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.002+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 >> 172.21.15.88:0/1829223387 conn(0x7fc55009e4e0 msgr2=0x7fc5500d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.003+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 shutdown_connections 2024-04-26T20:01:33.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.003+0000 7fc557200640 1 -- 172.21.15.88:0/1829223387 wait complete. 2024-04-26T20:01:33.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:01:33.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.223+0000 7f021d200640 1 Processor -- start 2024-04-26T20:01:33.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.223+0000 7f021d200640 1 -- start start 2024-04-26T20:01:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.224+0000 7f021d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 0x7f021806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.224+0000 7f021d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.224+0000 7f021d200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02180af800 con 0x7f02180add80 2024-04-26T20:01:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.224+0000 7f021d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f02180ae340 con 0x7f021815d100 2024-04-26T20:01:33.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.224+0000 7f021d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f02180afaa0 con 0x7f02180b5ca0 2024-04-26T20:01:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.225+0000 7f0215400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.225+0000 7f0215e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 0x7f021806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.225+0000 7f0215400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:50718/0 (socket says 172.21.15.88:50718) 2024-04-26T20:01:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.225+0000 7f0215400640 1 -- 172.21.15.88:0/1175734980 learned_addr learned my addr 172.21.15.88:0/1175734980 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.226+0000 7f0215400640 1 -- 172.21.15.88:0/1175734980 >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 msgr2=0x7f021806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.226+0000 7f0215400640 1 --2- 172.21.15.88:0/1175734980 >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 0x7f021806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.226+0000 7f0215400640 1 -- 172.21.15.88:0/1175734980 >> v1:172.21.15.132:6789/0 conn(0x7f02180add80 legacy=0x7f0218072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.226+0000 7f0215400640 1 -- 172.21.15.88:0/1175734980 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02180afd80 con 0x7f02180b5ca0 2024-04-26T20:01:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.226+0000 7f0215e00640 1 --2- 172.21.15.88:0/1175734980 >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 0x7f021806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:01:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.227+0000 7f0215400640 1 --2- 172.21.15.88:0/1175734980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0208009870 tx=0x7f020802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1b28a4efa7e6db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.228+0000 7f0214a00640 1 -- 172.21.15.88:0/1175734980 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208037070 con 0x7f02180b5ca0 2024-04-26T20:01:33.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.228+0000 7f0214a00640 1 -- 172.21.15.88:0/1175734980 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0208032440 con 0x7f02180b5ca0 2024-04-26T20:01:33.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.228+0000 7f0214a00640 1 -- 172.21.15.88:0/1175734980 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208032810 con 0x7f02180b5ca0 2024-04-26T20:01:33.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.228+0000 7f021d200640 1 -- 172.21.15.88:0/1175734980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 msgr2=0x7f021806ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.228+0000 7f021d200640 1 --2- 172.21.15.88:0/1175734980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0208009870 tx=0x7f020802a180 comp rx=0 tx=0).stop 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 -- 172.21.15.88:0/1175734980 shutdown_connections 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 --2- 172.21.15.88:0/1175734980 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180b5ca0 0x7f021806ccb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 --2- 172.21.15.88:0/1175734980 >> v2:172.21.15.132:3300/0 conn(0x7f021815d100 0x7f021806c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 -- 172.21.15.88:0/1175734980 >> 172.21.15.88:0/1175734980 conn(0x7f02180b2d50 msgr2=0x7f02180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 -- 172.21.15.88:0/1175734980 shutdown_connections 2024-04-26T20:01:33.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.229+0000 7f021d200640 1 -- 172.21.15.88:0/1175734980 wait complete. 2024-04-26T20:01:33.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.230+0000 7f021d200640 1 Processor -- start 2024-04-26T20:01:33.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 -- start start 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180add80 0x7f021806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 0x7f021814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f021809ee50 con 0x7f02180b5ca0 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f02180b0340 con 0x7f021815d100 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.231+0000 7f021d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0218160b70 con 0x7f02180add80 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57972/0 (socket says 172.21.15.88:57972) 2024-04-26T20:01:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 -- 172.21.15.88:0/1234232313 learned_addr learned my addr 172.21.15.88:0/1234232313 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215e00640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180add80 0x7f021806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 -- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180add80 msgr2=0x7f021806f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180add80 0x7f021806f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0216800640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 0x7f021814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 -- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 msgr2=0x7f021814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 0x7f021814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0215400640 1 -- 172.21.15.88:0/1234232313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021809e5d0 con 0x7f02180b5ca0 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.232+0000 7f0216800640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 0x7f021814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f0215400640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0208009870 tx=0x7f02080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208041070 con 0x7f02180b5ca0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020803c890 con 0x7f02180b5ca0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021814f6e0 con 0x7f02180b5ca0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208037040 con 0x7f02180b5ca0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.233+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021814fca0 con 0x7f02180b5ca0 2024-04-26T20:01:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.234+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0218069650 con 0x7f02180b5ca0 2024-04-26T20:01:33.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.235+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0208044070 con 0x7f02180b5ca0 2024-04-26T20:01:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.236+0000 7f020e000640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.236+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f02080ac640 con 0x7f02180b5ca0 2024-04-26T20:01:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.236+0000 7f0215e00640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.237+0000 7f0215e00640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 0x7f01f406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0200009560 tx=0x7f0200008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.241+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f020807ab20 con 0x7f02180b5ca0 2024-04-26T20:01:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.442+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f02180d1940 con 0x7f02180b5ca0 2024-04-26T20:01:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.443+0000 7f020e000640 1 -- 172.21.15.88:0/1234232313 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f020807a420 con 0x7f02180b5ca0 2024-04-26T20:01:33.445 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:01:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 msgr2=0x7f01f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 0x7f01f406edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0200009560 tx=0x7f0200008040 comp rx=0 tx=0).stop 2024-04-26T20:01:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 msgr2=0x7f0218158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0208009870 tx=0x7f02080046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 shutdown_connections 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f021815d100 0x7f021814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02180b5ca0 0x7f0218158000 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f01f406c8f0 0x7f01f406edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 --2- 172.21.15.88:0/1234232313 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02180add80 0x7f021806f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 >> 172.21.15.88:0/1234232313 conn(0x7f02180b2d50 msgr2=0x7f02180b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.446+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 shutdown_connections 2024-04-26T20:01:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.447+0000 7f021d200640 1 -- 172.21.15.88:0/1234232313 wait complete. 2024-04-26T20:01:33.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:01:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.671+0000 7f0923200640 1 Processor -- start 2024-04-26T20:01:33.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.671+0000 7f0923200640 1 -- start start 2024-04-26T20:01:33.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.672+0000 7f0923200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0924146680 0x7f09240b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0923200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09240d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0923200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0924146a60 con 0x7f09240d0e70 2024-04-26T20:01:33.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0923200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f092406e970 con 0x7f0924146680 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0923200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f09240d17f0 con 0x7f09240b4b30 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0921e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0924146680 0x7f09240b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0921e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0924146680 0x7f09240b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:34096/0 (socket says 172.21.15.88:34096) 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0921e00640 1 -- 172.21.15.88:0/1002192931 learned_addr learned my addr 172.21.15.88:0/1002192931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f0922800640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09240d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.673+0000 7f091be00640 1 --1- 172.21.15.88:0/1002192931 >> v1:172.21.15.132:6789/0 conn(0x7f09240d0e70 0x7f0924160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:56692/0 (socket says 172.21.15.88:56692) 2024-04-26T20:01:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.674+0000 7f0921e00640 1 -- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 msgr2=0x7f09240d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.674+0000 7f0921e00640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09240d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.674+0000 7f0921e00640 1 -- 172.21.15.88:0/1002192931 >> v1:172.21.15.132:6789/0 conn(0x7f09240d0e70 legacy=0x7f0924160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.674+0000 7f0921e00640 1 -- 172.21.15.88:0/1002192931 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092406dd60 con 0x7f0924146680 2024-04-26T20:01:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.674+0000 7f0922800640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09240d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:01:33.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.676+0000 7f0921e00640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0924146680 0x7f09240b5990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0914009870 tx=0x7f091402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b06e4feb7641e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.676+0000 7f0921400640 1 -- 172.21.15.88:0/1002192931 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914037070 con 0x7f0924146680 2024-04-26T20:01:33.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.676+0000 7f0921400640 1 -- 172.21.15.88:0/1002192931 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0914032440 con 0x7f0924146680 2024-04-26T20:01:33.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.676+0000 7f0921400640 1 -- 172.21.15.88:0/1002192931 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914032810 con 0x7f0924146680 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 -- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0924146680 msgr2=0x7f09240b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0924146680 0x7f09240b5990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0914009870 tx=0x7f091402a250 comp rx=0 tx=0).stop 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 -- 172.21.15.88:0/1002192931 shutdown_connections 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09240d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 --2- 172.21.15.88:0/1002192931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0924146680 0x7f09240b5990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 -- 172.21.15.88:0/1002192931 >> 172.21.15.88:0/1002192931 conn(0x7f092406c740 msgr2=0x7f092406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:33.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 -- 172.21.15.88:0/1002192931 shutdown_connections 2024-04-26T20:01:33.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.677+0000 7f0923200640 1 -- 172.21.15.88:0/1002192931 wait complete. 2024-04-26T20:01:33.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.679+0000 7f0923200640 1 Processor -- start 2024-04-26T20:01:33.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.679+0000 7f0923200640 1 -- start start 2024-04-26T20:01:33.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09241586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09240d0e70 0x7f092415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f092406e600 con 0x7f0924146680 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f092406e320 con 0x7f09240d0e70 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.680+0000 7f0923200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f09240cd6a0 con 0x7f09240b4b30 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57978/0 (socket says 172.21.15.88:57978) 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 -- 172.21.15.88:0/3023591684 learned_addr learned my addr 172.21.15.88:0/3023591684 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0921e00640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09241586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f091be00640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09240d0e70 0x7f092415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 -- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 msgr2=0x7f09241586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09241586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 -- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09240d0e70 msgr2=0x7f092415c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09240d0e70 0x7f092415c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 -- 172.21.15.88:0/3023591684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0924072aa0 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.681+0000 7f0922800640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f091c00efc0 tx=0x7f091c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.682+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c018020 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.682+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c014480 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.682+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0924072d80 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.682+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c014850 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.682+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0924073340 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.683+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0924073540 con 0x7f0924146680 2024-04-26T20:01:33.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.684+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f091c005d10 con 0x7f0924146680 2024-04-26T20:01:33.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.685+0000 7f091a000640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:33.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.685+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f091c08cb40 con 0x7f0924146680 2024-04-26T20:01:33.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.685+0000 7f0921e00640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:33.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.686+0000 7f0921e00640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 0x7f090006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:33.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.689+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0924073540 con 0x7f0924146680 2024-04-26T20:01:33.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.898+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09240693c0 con 0x7f0924146680 2024-04-26T20:01:33.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.899+0000 7f091a000640 1 -- 172.21.15.88:0/3023591684 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) ==== 80+0+122 (secure 0 0 0) 0x7f091c022030 con 0x7f0924146680 2024-04-26T20:01:33.901 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6837/2458488289 2024-04-27T20:01:28.259287+0000 2024-04-26T20:01:33.901 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6835/2458488289 2024-04-27T20:01:28.259287+0000 2024-04-26T20:01:33.901 INFO:teuthology.orchestra.run.smithi088.stderr:listed 2 entries 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 msgr2=0x7f090006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 0x7f090006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 msgr2=0x7f092415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f091c00efc0 tx=0x7f091c00c490 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 shutdown_connections 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0924146680 0x7f092415a4d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09240d0e70 0x7f092415c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f090006c8f0 0x7f090006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 --2- 172.21.15.88:0/3023591684 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f09240b4b30 0x7f09241586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 >> 172.21.15.88:0/3023591684 conn(0x7f092406c740 msgr2=0x7f09240cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:33.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 shutdown_connections 2024-04-26T20:01:33.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:33.902+0000 7f0923200640 1 -- 172.21.15.88:0/3023591684 wait complete. 2024-04-26T20:01:33.919 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-04-26T20:01:33.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:33.920 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:01:33.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[47131]: fuse finished with error 0 and tester_r 0 2024-04-26T20:01:33.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:01:33.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:33.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:01:34.147 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:01:34.147 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:01:34.147 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:34.205 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:01:34.205 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:34.261 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:34.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.461+0000 7fdd71800640 1 Processor -- start 2024-04-26T20:01:34.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.461+0000 7fdd71800640 1 -- start start 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd71800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cb70 0x7fdd6c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd71800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdd6c15cfd0 0x7fdd6c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd71800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd6c0ae850 con 0x7fdd6c06c770 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd71800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdd6c0af460 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd71800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6c0aeb30 con 0x7fdd6c15cb70 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd6aa00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdd6c06c770 0x7fdd6c06cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55558/0 (socket says 172.21.15.88:55558) 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd6aa00640 1 -- 172.21.15.88:0/779335805 learned_addr learned my addr 172.21.15.88:0/779335805 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:34.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.463+0000 7fdd6b400640 1 --2- 172.21.15.88:0/779335805 >> v2:172.21.15.88:3300/0 conn(0x7fdd6c15cfd0 0x7fdd6c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd6a000640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cb70 0x7fdd6c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd69600640 1 -- 172.21.15.88:0/779335805 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037349361 0 0) 0x7fdd6c0ae850 con 0x7fdd6c06c770 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd6b400640 1 -- 172.21.15.88:0/779335805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cb70 msgr2=0x7fdd6c06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd6b400640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cb70 0x7fdd6c06ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd6b400640 1 -- 172.21.15.88:0/779335805 >> v1:172.21.15.88:6789/0 conn(0x7fdd6c06c770 legacy=0x7fdd6c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.464+0000 7fdd6b400640 1 -- 172.21.15.88:0/779335805 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c0aee10 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.465+0000 7fdd6b400640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cfd0 0x7fdd6c06f3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097a0 tx=0x7fdd6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d97edd14cb6f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.465+0000 7fdd69600640 1 -- 172.21.15.88:0/779335805 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64037070 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.465+0000 7fdd69600640 1 -- 172.21.15.88:0/779335805 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd64032440 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.465+0000 7fdd69600640 1 -- 172.21.15.88:0/779335805 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64032810 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 -- 172.21.15.88:0/779335805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cfd0 msgr2=0x7fdd6c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cfd0 0x7fdd6c06f3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097a0 tx=0x7fdd6402a180 comp rx=0 tx=0).stop 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 -- 172.21.15.88:0/779335805 shutdown_connections 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cfd0 0x7fdd6c06f3a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 --2- 172.21.15.88:0/779335805 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cb70 0x7fdd6c06ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 -- 172.21.15.88:0/779335805 >> 172.21.15.88:0/779335805 conn(0x7fdd6c0b2d50 msgr2=0x7fdd6c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 -- 172.21.15.88:0/779335805 shutdown_connections 2024-04-26T20:01:34.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.466+0000 7fdd71800640 1 -- 172.21.15.88:0/779335805 wait complete. 2024-04-26T20:01:34.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.468+0000 7fdd71800640 1 Processor -- start 2024-04-26T20:01:34.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.468+0000 7fdd71800640 1 -- start start 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06c770 0x7fdd6c09e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cfd0 0x7fdd6c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c15c780 con 0x7fdd6c15cb70 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c15c600 con 0x7fdd6c06c770 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd71800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6c0af8f0 con 0x7fdd6c15cfd0 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd6a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd6a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57994/0 (socket says 172.21.15.88:57994) 2024-04-26T20:01:34.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.469+0000 7fdd6a000640 1 -- 172.21.15.88:0/2747802538 learned_addr learned my addr 172.21.15.88:0/2747802538 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 -- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cfd0 msgr2=0x7fdd6c09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cfd0 0x7fdd6c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 -- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06c770 msgr2=0x7fdd6c09e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06c770 0x7fdd6c09e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 -- 172.21.15.88:0/2747802538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c09ea60 con 0x7fdd6c15cb70 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd6a000640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007f20 tx=0x7fdd5400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54014620 con 0x7fdd6c15cb70 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd54019070 con 0x7fdd6c15cb70 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54014a00 con 0x7fdd6c15cb70 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c152c20 con 0x7fdd6c15cb70 2024-04-26T20:01:34.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.470+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c0cac00 con 0x7fdd6c15cb70 2024-04-26T20:01:34.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.471+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6c069540 con 0x7fdd6c15cb70 2024-04-26T20:01:34.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.472+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd54014ba0 con 0x7fdd6c15cb70 2024-04-26T20:01:34.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.473+0000 7fdd62a00640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 0x7fdd4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.473+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fdd5408c520 con 0x7fdd6c15cb70 2024-04-26T20:01:34.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.474+0000 7fdd6aa00640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 0x7fdd4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.474+0000 7fdd6aa00640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 0x7fdd4806ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007fd0 tx=0x7fdd5c007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.478+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd5405a850 con 0x7fdd6c15cb70 2024-04-26T20:01:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.665+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd6c06fdc0 con 0x7fdd6c15cb70 2024-04-26T20:01:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.668+0000 7fdd62a00640 1 -- 172.21.15.88:0/2747802538 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16786 (secure 0 0 0) 0x7fdd5405a150 con 0x7fdd6c15cb70 2024-04-26T20:01:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:34.670 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:31.270527+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:34.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.670+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 msgr2=0x7fdd4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.670+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 0x7fdd4806ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c007fd0 tx=0x7fdd5c007920 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 msgr2=0x7fdd6c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007f20 tx=0x7fdd5400e570 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 shutdown_connections 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdd4806c920 0x7fdd4806ede0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdd6c15cfd0 0x7fdd6c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd6c15cb70 0x7fdd6c1526e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 --2- 172.21.15.88:0/2747802538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd6c06c770 0x7fdd6c09e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 >> 172.21.15.88:0/2747802538 conn(0x7fdd6c0b2d50 msgr2=0x7fdd6c072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 shutdown_connections 2024-04-26T20:01:34.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.671+0000 7fdd71800640 1 -- 172.21.15.88:0/2747802538 wait complete. 2024-04-26T20:01:34.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:03.091802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:01:04.331500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:34.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:34.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.892+0000 7f9195c00640 1 Processor -- start 2024-04-26T20:01:34.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.893+0000 7f9195c00640 1 -- start start 2024-04-26T20:01:34.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f9195c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f91900d19c0 0x7f91900d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f9195c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91900d3830 con 0x7f91900d1da0 2024-04-26T20:01:34.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f9195c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91900d3b10 con 0x7f91900b6170 2024-04-26T20:01:34.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f9195c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f91900d40d0 con 0x7f91900d19c0 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f918ea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f91900d19c0 0x7f91900d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f918f400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f91900d1da0 0x7f9190162640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55572/0 (socket says 172.21.15.88:55572) 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.894+0000 7f918f400640 1 -- 172.21.15.88:0/2021753429 learned_addr learned my addr 172.21.15.88:0/2021753429 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.895+0000 7f918ea00640 1 -- 172.21.15.88:0/2021753429 >> v1:172.21.15.132:6789/0 conn(0x7f91900b6170 legacy=0x7f91900c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.895+0000 7f918ea00640 1 -- 172.21.15.88:0/2021753429 >> v1:172.21.15.88:6789/0 conn(0x7f91900d1da0 legacy=0x7f9190162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.895+0000 7f918ea00640 1 -- 172.21.15.88:0/2021753429 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91900d3df0 con 0x7f91900d19c0 2024-04-26T20:01:34.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f918ea00640 1 --2- 172.21.15.88:0/2021753429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d19c0 0x7f91900d2f00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91800097a0 tx=0x7f918002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6e0a33910a5e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f918e000640 1 -- 172.21.15.88:0/2021753429 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180037070 con 0x7f91900d19c0 2024-04-26T20:01:34.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f918e000640 1 -- 172.21.15.88:0/2021753429 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918002ae50 con 0x7f91900d19c0 2024-04-26T20:01:34.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f918e000640 1 -- 172.21.15.88:0/2021753429 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180032660 con 0x7f91900d19c0 2024-04-26T20:01:34.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f9195c00640 1 -- 172.21.15.88:0/2021753429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d19c0 msgr2=0x7f91900d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.896+0000 7f9195c00640 1 --2- 172.21.15.88:0/2021753429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d19c0 0x7f91900d2f00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f91800097a0 tx=0x7f918002a140 comp rx=0 tx=0).stop 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.897+0000 7f9195c00640 1 -- 172.21.15.88:0/2021753429 shutdown_connections 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.897+0000 7f9195c00640 1 --2- 172.21.15.88:0/2021753429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d19c0 0x7f91900d2f00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.897+0000 7f9195c00640 1 -- 172.21.15.88:0/2021753429 >> 172.21.15.88:0/2021753429 conn(0x7f919009e460 msgr2=0x7f91900d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.897+0000 7f9195c00640 1 -- 172.21.15.88:0/2021753429 shutdown_connections 2024-04-26T20:01:34.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.897+0000 7f9195c00640 1 -- 172.21.15.88:0/2021753429 wait complete. 2024-04-26T20:01:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.898+0000 7f9195c00640 1 Processor -- start 2024-04-26T20:01:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.898+0000 7f9195c00640 1 -- start start 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91900b6170 0x7f91900b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 0x7f91901f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9190162d50 con 0x7f91900d1da0 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f91900cfc30 con 0x7f91900d19c0 2024-04-26T20:01:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.899+0000 7f9195c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9190162ed0 con 0x7f91900b6170 2024-04-26T20:01:34.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58020/0 (socket says 172.21.15.88:58020) 2024-04-26T20:01:34.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 -- 172.21.15.88:0/3307089865 learned_addr learned my addr 172.21.15.88:0/3307089865 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f9187e00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 0x7f91901f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918ea00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91900b6170 0x7f91900b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 -- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91900b6170 msgr2=0x7f91900b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91900b6170 0x7f91900b6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 -- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 msgr2=0x7f91901f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 0x7f91901f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 -- 172.21.15.88:0/3307089865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91900a2800 con 0x7f91900d1da0 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f9187e00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 0x7f91901f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.900+0000 7f918f400640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f918800cbc0 tx=0x7f918800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.901+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188017070 con 0x7f91900d1da0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.901+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918800ad40 con 0x7f91900d1da0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.901+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188012600 con 0x7f91900d1da0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.901+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901f9990 con 0x7f91900d1da0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.901+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91900a2a80 con 0x7f91900d1da0 2024-04-26T20:01:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.902+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9190069650 con 0x7f91900d1da0 2024-04-26T20:01:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.903+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f918801a070 con 0x7f91900d1da0 2024-04-26T20:01:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.904+0000 7f9186a00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 0x7f916c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.904+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f918808b890 con 0x7f91900d1da0 2024-04-26T20:01:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.905+0000 7f918ea00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 0x7f916c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.905+0000 7f918ea00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 0x7f916c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9180034000 tx=0x7f918002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:34.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:34.909+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91901f9990 con 0x7f91900d1da0 2024-04-26T20:01:35.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.109+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91900c9380 con 0x7f91900d1da0 2024-04-26T20:01:35.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.110+0000 7f9186a00640 1 -- 172.21.15.88:0/3307089865 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f91900a2800 con 0x7f91900d1da0 2024-04-26T20:01:35.111 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:35.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 msgr2=0x7f916c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 0x7f916c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9180034000 tx=0x7f918002e000 comp rx=0 tx=0).stop 2024-04-26T20:01:35.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 msgr2=0x7f91900a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f918800cbc0 tx=0x7f918800a430 comp rx=0 tx=0).stop 2024-04-26T20:01:35.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 shutdown_connections 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91900d1da0 0x7f91900a07f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91900d19c0 0x7f91901f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f916c06c8f0 0x7f916c06edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 --2- 172.21.15.88:0/3307089865 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f91900b6170 0x7f91900b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.112+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 >> 172.21.15.88:0/3307089865 conn(0x7f919009e460 msgr2=0x7f91900c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.113+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 shutdown_connections 2024-04-26T20:01:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.113+0000 7f9195c00640 1 -- 172.21.15.88:0/3307089865 wait complete. 2024-04-26T20:01:35.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:01:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.318+0000 7f40b4c00640 1 Processor -- start 2024-04-26T20:01:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.318+0000 7f40b4c00640 1 -- start start 2024-04-26T20:01:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40b4c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f40b00721b0 0x7f40b0163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40b4c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b006e080 con 0x7f40b014cef0 2024-04-26T20:01:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40b4c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b006e360 con 0x7f40b01640f0 2024-04-26T20:01:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40b4c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f40b006e640 con 0x7f40b00721b0 2024-04-26T20:01:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40af400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f40b014cef0 0x7f40b00730a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55582/0 (socket says 172.21.15.88:55582) 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40af400640 1 -- 172.21.15.88:0/3451400613 learned_addr learned my addr 172.21.15.88:0/3451400613 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.320+0000 7f40a7e00640 1 --2- 172.21.15.88:0/3451400613 >> v2:172.21.15.88:3300/0 conn(0x7f40b00721b0 0x7f40b0163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.321+0000 7f40aea00640 1 -- 172.21.15.88:0/3451400613 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294060613 0 0) 0x7f40b006e080 con 0x7f40b014cef0 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.321+0000 7f40aea00640 1 -- 172.21.15.88:0/3451400613 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4094003520 con 0x7f40b014cef0 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.321+0000 7f40a7e00640 1 -- 172.21.15.88:0/3451400613 >> v1:172.21.15.132:6789/0 conn(0x7f40b01640f0 legacy=0x7f40b00b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.321+0000 7f40a7e00640 1 -- 172.21.15.88:0/3451400613 >> v1:172.21.15.88:6789/0 conn(0x7f40b014cef0 legacy=0x7f40b00730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.321+0000 7f40a7e00640 1 -- 172.21.15.88:0/3451400613 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b014d2d0 con 0x7f40b00721b0 2024-04-26T20:01:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.322+0000 7f40a7e00640 1 --2- 172.21.15.88:0/3451400613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b00721b0 0x7f40b0163bb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f40980097a0 tx=0x7f409802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8c9dc03f60e42c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.322+0000 7f40aea00640 1 -- 172.21.15.88:0/3451400613 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4098037070 con 0x7f40b00721b0 2024-04-26T20:01:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.322+0000 7f40aea00640 1 -- 172.21.15.88:0/3451400613 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4098032440 con 0x7f40b00721b0 2024-04-26T20:01:35.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.322+0000 7f40aea00640 1 -- 172.21.15.88:0/3451400613 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40980327f0 con 0x7f40b00721b0 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 -- 172.21.15.88:0/3451400613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b00721b0 msgr2=0x7f40b0163bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 --2- 172.21.15.88:0/3451400613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b00721b0 0x7f40b0163bb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f40980097a0 tx=0x7f409802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 -- 172.21.15.88:0/3451400613 shutdown_connections 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 --2- 172.21.15.88:0/3451400613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b00721b0 0x7f40b0163bb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 -- 172.21.15.88:0/3451400613 >> 172.21.15.88:0/3451400613 conn(0x7f40b00b2900 msgr2=0x7f40b00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 -- 172.21.15.88:0/3451400613 shutdown_connections 2024-04-26T20:01:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.323+0000 7f40b4c00640 1 -- 172.21.15.88:0/3451400613 wait complete. 2024-04-26T20:01:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.324+0000 7f40b4c00640 1 Processor -- start 2024-04-26T20:01:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.325+0000 7f40b4c00640 1 -- start start 2024-04-26T20:01:35.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.325+0000 7f40b4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 0x7f40b01626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40b4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40b014cef0 0x7f40b01fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40b4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40b4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b006ee40 con 0x7f40b01640f0 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40b4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f40b00b4560 con 0x7f40b014cef0 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40b4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f40b006efc0 con 0x7f40b00721b0 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58038/0 (socket says 172.21.15.88:58038) 2024-04-26T20:01:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 -- 172.21.15.88:0/1020679280 learned_addr learned my addr 172.21.15.88:0/1020679280 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40af400640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 0x7f40b01626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40a7e00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40b014cef0 0x7f40b01fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 -- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 msgr2=0x7f40b01626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 0x7f40b01626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 -- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40b014cef0 msgr2=0x7f40b01fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40b014cef0 0x7f40b01fa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40afe00640 1 -- 172.21.15.88:0/1020679280 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b0146b10 con 0x7f40b01640f0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.326+0000 7f40af400640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 0x7f40b01626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40afe00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40a800be60 tx=0x7f40a800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8019070 con 0x7f40b01640f0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8014440 con 0x7f40b01640f0 2024-04-26T20:01:35.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014830 con 0x7f40b01640f0 2024-04-26T20:01:35.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b0146df0 con 0x7f40b01640f0 2024-04-26T20:01:35.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.327+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b0162be0 con 0x7f40b01640f0 2024-04-26T20:01:35.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.328+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b00693c0 con 0x7f40b01640f0 2024-04-26T20:01:35.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.329+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40a801c070 con 0x7f40b01640f0 2024-04-26T20:01:35.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.330+0000 7f40a7400640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 0x7f408c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.330+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f40a808ba80 con 0x7f40b01640f0 2024-04-26T20:01:35.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.330+0000 7f40af400640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 0x7f408c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.331+0000 7f40af400640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 0x7f408c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40a0005880 tx=0x7f40a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.335+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40a808de20 con 0x7f40b01640f0 2024-04-26T20:01:35.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.533+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f40b00b55b0 con 0x7f40b01640f0 2024-04-26T20:01:35.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.534+0000 7f40a7400640 1 -- 172.21.15.88:0/1020679280 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f40a8059e40 con 0x7f40b01640f0 2024-04-26T20:01:35.536 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 msgr2=0x7f408c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 0x7f408c06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40a0005880 tx=0x7f40a0009040 comp rx=0 tx=0).stop 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 msgr2=0x7f40b01eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40a800be60 tx=0x7f40a800bf50 comp rx=0 tx=0).stop 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 shutdown_connections 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b01640f0 0x7f40b01eae30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40b014cef0 0x7f40b01fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f408c06c8f0 0x7f408c06edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 --2- 172.21.15.88:0/1020679280 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f40b00721b0 0x7f40b01626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 >> 172.21.15.88:0/1020679280 conn(0x7f40b00b2900 msgr2=0x7f40b0162f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 shutdown_connections 2024-04-26T20:01:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.537+0000 7f40b4c00640 1 -- 172.21.15.88:0/1020679280 wait complete. 2024-04-26T20:01:35.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:01:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.759+0000 7f003d200640 1 Processor -- start 2024-04-26T20:01:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.760+0000 7f003d200640 1 -- start start 2024-04-26T20:01:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f003d200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f00380d19c0 0x7f00380d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f003d200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f00380d1da0 0x7f0038162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f003d200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00380d3830 con 0x7f00380b6170 2024-04-26T20:01:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f003d200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f00380d40d0 con 0x7f00380d19c0 2024-04-26T20:01:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f003d200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f00380d3b10 con 0x7f00380d1da0 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.761+0000 7f0035e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f00380d19c0 0x7f00380d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f00380d19c0 0x7f00380d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58052/0 (socket says 172.21.15.88:58052) 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 -- 172.21.15.88:0/3587695467 learned_addr learned my addr 172.21.15.88:0/3587695467 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0036800640 1 --2- 172.21.15.88:0/3587695467 >> v2:172.21.15.132:3300/0 conn(0x7f00380d1da0 0x7f0038162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 -- 172.21.15.88:0/3587695467 >> v2:172.21.15.132:3300/0 conn(0x7f00380d1da0 msgr2=0x7f0038162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 --2- 172.21.15.88:0/3587695467 >> v2:172.21.15.132:3300/0 conn(0x7f00380d1da0 0x7f0038162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 -- 172.21.15.88:0/3587695467 >> v1:172.21.15.132:6789/0 conn(0x7f00380b6170 legacy=0x7f00380c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.762+0000 7f0035e00640 1 -- 172.21.15.88:0/3587695467 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00380d3df0 con 0x7f00380d19c0 2024-04-26T20:01:35.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.763+0000 7f0035e00640 1 --2- 172.21.15.88:0/3587695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380d19c0 0x7f00380d2f00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f003808e590 tx=0x7f002802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e6ee5518b820a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.763+0000 7f0035400640 1 -- 172.21.15.88:0/3587695467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028037070 con 0x7f00380d19c0 2024-04-26T20:01:35.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.763+0000 7f0035400640 1 -- 172.21.15.88:0/3587695467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0028032440 con 0x7f00380d19c0 2024-04-26T20:01:35.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.763+0000 7f0035400640 1 -- 172.21.15.88:0/3587695467 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028032810 con 0x7f00380d19c0 2024-04-26T20:01:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 -- 172.21.15.88:0/3587695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380d19c0 msgr2=0x7f00380d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 --2- 172.21.15.88:0/3587695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380d19c0 0x7f00380d2f00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f003808e590 tx=0x7f002802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 -- 172.21.15.88:0/3587695467 shutdown_connections 2024-04-26T20:01:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 --2- 172.21.15.88:0/3587695467 >> v2:172.21.15.132:3300/0 conn(0x7f00380d1da0 0x7f0038162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 --2- 172.21.15.88:0/3587695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380d19c0 0x7f00380d2f00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 -- 172.21.15.88:0/3587695467 >> 172.21.15.88:0/3587695467 conn(0x7f003809e460 msgr2=0x7f00380d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 -- 172.21.15.88:0/3587695467 shutdown_connections 2024-04-26T20:01:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.764+0000 7f003d200640 1 -- 172.21.15.88:0/3587695467 wait complete. 2024-04-26T20:01:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.766+0000 7f003d200640 1 Processor -- start 2024-04-26T20:01:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.766+0000 7f003d200640 1 -- start start 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00380d19c0 0x7f0038159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00380d1da0 0x7f0038155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0038162b80 con 0x7f00380b6170 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f00380cfc30 con 0x7f00380d19c0 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f003d200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0038162d00 con 0x7f00380d1da0 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f0035e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f0035e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58064/0 (socket says 172.21.15.88:58064) 2024-04-26T20:01:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.767+0000 7f0035e00640 1 -- 172.21.15.88:0/1555083060 learned_addr learned my addr 172.21.15.88:0/1555083060 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:35.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 -- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00380d1da0 msgr2=0x7f0038155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00380d1da0 0x7f0038155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 -- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00380d19c0 msgr2=0x7f0038159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00380d19c0 0x7f0038159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 -- 172.21.15.88:0/1555083060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00381f9a80 con 0x7f00380b6170 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f0035e00640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0028000c00 tx=0x7f00280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028041020 con 0x7f00380b6170 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00280306c0 con 0x7f00380b6170 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028037040 con 0x7f00380b6170 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.768+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00381f6470 con 0x7f00380b6170 2024-04-26T20:01:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.769+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00381f5c70 con 0x7f00380b6170 2024-04-26T20:01:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.770+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00381f57c0 con 0x7f00380b6170 2024-04-26T20:01:35.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.771+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0028044020 con 0x7f00380b6170 2024-04-26T20:01:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.772+0000 7f002e000640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.772+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f00280ac450 con 0x7f00380b6170 2024-04-26T20:01:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.772+0000 7f002fe00640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.773+0000 7f002fe00640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0020001560 tx=0x7f0020007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:35.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.776+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f002804b030 con 0x7f00380b6170 2024-04-26T20:01:35.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.974+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f00380d22f0 con 0x7f00380b6170 2024-04-26T20:01:35.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.975+0000 7f002e000640 1 -- 172.21.15.88:0/1555083060 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f002807a780 con 0x7f00380b6170 2024-04-26T20:01:35.977 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 msgr2=0x7f001406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0020001560 tx=0x7f0020007990 comp rx=0 tx=0).stop 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 msgr2=0x7f00380b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0028000c00 tx=0x7f00280046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 shutdown_connections 2024-04-26T20:01:35.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f00380d1da0 0x7f0038155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00380d19c0 0x7f0038159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 --2- 172.21.15.88:0/1555083060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00380b6170 0x7f00380b5850 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 >> 172.21.15.88:0/1555083060 conn(0x7f003809e460 msgr2=0x7f00380c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 shutdown_connections 2024-04-26T20:01:35.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:35.978+0000 7f003d200640 1 -- 172.21.15.88:0/1555083060 wait complete. 2024-04-26T20:01:35.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:01:36.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.196+0000 7f958c600640 1 Processor -- start 2024-04-26T20:01:36.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.197+0000 7f958c600640 1 -- start start 2024-04-26T20:01:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.198+0000 7f958c600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958406c770 0x7f958406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.198+0000 7f958c600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f958415cb70 0x7f9584073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.198+0000 7f958c600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958406d090 con 0x7f958415cfd0 2024-04-26T20:01:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.198+0000 7f958c600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f95840af460 con 0x7f958415cb70 2024-04-26T20:01:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.198+0000 7f958c600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f95840ae850 con 0x7f958406c770 2024-04-26T20:01:36.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.199+0000 7f9589200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f958415cb70 0x7f9584073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.199+0000 7f958a600640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f958415cfd0 0x7f9584073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:56718/0 (socket says 172.21.15.88:56718) 2024-04-26T20:01:36.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.199+0000 7f958a600640 1 -- 172.21.15.88:0/1907804672 learned_addr learned my addr 172.21.15.88:0/1907804672 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:36.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.199+0000 7f9589c00640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958406c770 0x7f958406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.200+0000 7f9589200640 1 -- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958406c770 msgr2=0x7f958406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:36.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.200+0000 7f9589200640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958406c770 0x7f958406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.200+0000 7f9589200640 1 -- 172.21.15.88:0/1907804672 >> v1:172.21.15.132:6789/0 conn(0x7f958415cfd0 legacy=0x7f9584073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.200+0000 7f9589200640 1 -- 172.21.15.88:0/1907804672 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840aeb30 con 0x7f958415cb70 2024-04-26T20:01:36.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.201+0000 7f9589200640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958415cb70 0x7f9584073120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed75bb8de305a6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f9583e00640 1 -- 172.21.15.88:0/1907804672 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f958415cb70 2024-04-26T20:01:36.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f9583e00640 1 -- 172.21.15.88:0/1907804672 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570032440 con 0x7f958415cb70 2024-04-26T20:01:36.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f9583e00640 1 -- 172.21.15.88:0/1907804672 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570032810 con 0x7f958415cb70 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 -- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958415cb70 msgr2=0x7f9584073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958415cb70 0x7f9584073120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a180 comp rx=0 tx=0).stop 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 -- 172.21.15.88:0/1907804672 shutdown_connections 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958415cb70 0x7f9584073120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 --2- 172.21.15.88:0/1907804672 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958406c770 0x7f958406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 -- 172.21.15.88:0/1907804672 >> 172.21.15.88:0/1907804672 conn(0x7f95840b2d50 msgr2=0x7f95840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 -- 172.21.15.88:0/1907804672 shutdown_connections 2024-04-26T20:01:36.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.202+0000 7f958c600640 1 -- 172.21.15.88:0/1907804672 wait complete. 2024-04-26T20:01:36.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.204+0000 7f958c600640 1 Processor -- start 2024-04-26T20:01:36.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.204+0000 7f958c600640 1 -- start start 2024-04-26T20:01:36.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958406c770 0x7f95841551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958415cfd0 0x7f958415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f958415c600 con 0x7f958415cb70 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f95840af0f0 con 0x7f958406c770 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f958c600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f958415c780 con 0x7f958415cfd0 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f9589200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f9589200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58072/0 (socket says 172.21.15.88:58072) 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.205+0000 7f9589200640 1 -- 172.21.15.88:0/3398419573 learned_addr learned my addr 172.21.15.88:0/3398419573 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 -- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958415cfd0 msgr2=0x7f958415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958415cfd0 0x7f958415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 -- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958406c770 msgr2=0x7f95841551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958406c770 0x7f95841551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 -- 172.21.15.88:0/3398419573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95841568c0 con 0x7f958415cb70 2024-04-26T20:01:36.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9589200640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f957002a1c0 tx=0x7f95700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.206+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570041020 con 0x7f958415cb70 2024-04-26T20:01:36.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.207+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570030700 con 0x7f958415cb70 2024-04-26T20:01:36.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.207+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037040 con 0x7f958415cb70 2024-04-26T20:01:36.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.207+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9584156ba0 con 0x7f958415cb70 2024-04-26T20:01:36.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.207+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9584157160 con 0x7f958415cb70 2024-04-26T20:01:36.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.208+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9584069540 con 0x7f958415cb70 2024-04-26T20:01:36.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.209+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9570044020 con 0x7f958415cb70 2024-04-26T20:01:36.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.210+0000 7f9581600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 0x7f956406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.210+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f95700ac450 con 0x7f958415cb70 2024-04-26T20:01:36.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.210+0000 7f9589c00640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 0x7f956406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.211+0000 7f9589c00640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 0x7f956406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9574006a90 tx=0x7f9574006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.214+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f957004b030 con 0x7f958415cb70 2024-04-26T20:01:36.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.425+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95840cbe20 con 0x7f958415cb70 2024-04-26T20:01:36.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.426+0000 7f9581600640 1 -- 172.21.15.88:0/3398419573 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) ==== 80+0+122 (secure 0 0 0) 0x7f957007a780 con 0x7f958415cb70 2024-04-26T20:01:36.427 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6837/2458488289 2024-04-27T20:01:28.259287+0000 2024-04-26T20:01:36.428 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6835/2458488289 2024-04-27T20:01:28.259287+0000 2024-04-26T20:01:36.428 INFO:teuthology.orchestra.run.smithi088.stderr:listed 2 entries 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 msgr2=0x7f956406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 0x7f956406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9574006a90 tx=0x7f9574006a20 comp rx=0 tx=0).stop 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 msgr2=0x7f9584155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f957002a1c0 tx=0x7f95700046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 shutdown_connections 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f956406c8f0 0x7f956406edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f958415cfd0 0x7f958415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f958415cb70 0x7f9584155f50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 --2- 172.21.15.88:0/3398419573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f958406c770 0x7f95841551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 >> 172.21.15.88:0/3398419573 conn(0x7f95840b2d50 msgr2=0x7f95840cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:36.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 shutdown_connections 2024-04-26T20:01:36.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.428+0000 7f958c600640 1 -- 172.21.15.88:0/3398419573 wait complete. 2024-04-26T20:01:36.449 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi132.front.sepia.ceph.com... 2024-04-26T20:01:36.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:36.449 DEBUG:teuthology.orchestra.run.smithi132:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:01:36.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40036]: fuse finished with error 0 and tester_r 0 2024-04-26T20:01:36.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:01:36.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:01:36.541 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:01:36.641 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:01:36.641 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:01:36.641 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:36.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:01:36.696 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:01:36.697 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:01:36.697 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:36.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:01:36.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:36.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.965+0000 7fddf4200640 1 Processor -- start 2024-04-26T20:01:36.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.965+0000 7fddf4200640 1 -- start start 2024-04-26T20:01:36.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.967+0000 7fddf4200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 0x7fddec1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.967+0000 7fddf4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 0x7fddec06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.967+0000 7fddf4200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddec0b5fb0 con 0x7fddec06c770 2024-04-26T20:01:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.967+0000 7fddf4200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fddec159c80 con 0x7fddec1598a0 2024-04-26T20:01:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.967+0000 7fddf4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fddec06d110 con 0x7fddec15fe50 2024-04-26T20:01:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.968+0000 7fddf1800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 0x7fddec1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.968+0000 7fddf2200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 0x7fddec06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.968+0000 7fddf0e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fddec06c770 0x7fddec15f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:56734/0 (socket says 172.21.15.88:56734) 2024-04-26T20:01:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.968+0000 7fddf0e00640 1 -- 172.21.15.88:0/3778210487 learned_addr learned my addr 172.21.15.88:0/3778210487 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:36.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.969+0000 7fddf2200640 1 -- 172.21.15.88:0/3778210487 >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 msgr2=0x7fddec1638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.969+0000 7fddf2200640 1 --2- 172.21.15.88:0/3778210487 >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 0x7fddec1638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.969+0000 7fddf2200640 1 -- 172.21.15.88:0/3778210487 >> v1:172.21.15.132:6789/0 conn(0x7fddec06c770 legacy=0x7fddec15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.969+0000 7fddf2200640 1 -- 172.21.15.88:0/3778210487 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec160370 con 0x7fddec15fe50 2024-04-26T20:01:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.969+0000 7fddf1800640 1 --2- 172.21.15.88:0/3778210487 >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 0x7fddec1638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:01:36.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.970+0000 7fddf2200640 1 --2- 172.21.15.88:0/3778210487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 0x7fddec06cbd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdde40097a0 tx=0x7fdde402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add852b172db7b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddebe00640 1 -- 172.21.15.88:0/3778210487 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4037070 con 0x7fddec15fe50 2024-04-26T20:01:36.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddebe00640 1 -- 172.21.15.88:0/3778210487 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde402ae50 con 0x7fddec15fe50 2024-04-26T20:01:36.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddebe00640 1 -- 172.21.15.88:0/3778210487 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4032660 con 0x7fddec15fe50 2024-04-26T20:01:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 -- 172.21.15.88:0/3778210487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 msgr2=0x7fddec06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 --2- 172.21.15.88:0/3778210487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 0x7fddec06cbd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdde40097a0 tx=0x7fdde402a100 comp rx=0 tx=0).stop 2024-04-26T20:01:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 -- 172.21.15.88:0/3778210487 shutdown_connections 2024-04-26T20:01:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 --2- 172.21.15.88:0/3778210487 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec15fe50 0x7fddec06cbd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 --2- 172.21.15.88:0/3778210487 >> v2:172.21.15.132:3300/0 conn(0x7fddec1598a0 0x7fddec1638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.971+0000 7fddf4200640 1 -- 172.21.15.88:0/3778210487 >> 172.21.15.88:0/3778210487 conn(0x7fddec0b2d50 msgr2=0x7fddec0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:36.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.972+0000 7fddf4200640 1 -- 172.21.15.88:0/3778210487 shutdown_connections 2024-04-26T20:01:36.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.972+0000 7fddf4200640 1 -- 172.21.15.88:0/3778210487 wait complete. 2024-04-26T20:01:36.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.973+0000 7fddf4200640 1 Processor -- start 2024-04-26T20:01:36.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.973+0000 7fddf4200640 1 -- start start 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec06c770 0x7fddec146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddec15fe50 0x7fddec0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddec163390 con 0x7fddec1598a0 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fddec1630b0 con 0x7fddec15fe50 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.974+0000 7fddf4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fddec160650 con 0x7fddec06c770 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58082/0 (socket says 172.21.15.88:58082) 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 -- 172.21.15.88:0/704089270 learned_addr learned my addr 172.21.15.88:0/704089270 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 -- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec06c770 msgr2=0x7fddec146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf1800640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec06c770 0x7fddec146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf2200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddec15fe50 0x7fddec0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec06c770 0x7fddec146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 -- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddec15fe50 msgr2=0x7fddec0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddec15fe50 0x7fddec0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 -- 172.21.15.88:0/704089270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec1f5040 con 0x7fddec1598a0 2024-04-26T20:01:36.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.975+0000 7fddf0e00640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdde000cb10 tx=0x7fdde000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.976+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0017070 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.976+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde000ac70 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.976+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0012620 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.976+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec0cc340 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.976+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec1f55c0 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.977+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec069540 con 0x7fddec1598a0 2024-04-26T20:01:36.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.978+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdde001a070 con 0x7fddec1598a0 2024-04-26T20:01:36.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.979+0000 7fdde9600640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 0x7fddd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:36.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.979+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fdde008ba40 con 0x7fddec1598a0 2024-04-26T20:01:36.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.980+0000 7fddf1800640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 0x7fddd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:36.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.980+0000 7fddf1800640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 0x7fddd006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009560 tx=0x7fdddc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:36.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:36.984+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdde0059d70 con 0x7fddec1598a0 2024-04-26T20:01:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.226+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddec073fc0 con 0x7fddec1598a0 2024-04-26T20:01:37.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.227+0000 7fdde9600640 1 -- 172.21.15.88:0/704089270 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7fdde0059670 con 0x7fddec1598a0 2024-04-26T20:01:37.228 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:37.229 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/809249379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":809249379},{"type":"v1","addr":"172.21.15.88:6837","nonce":809249379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:37.229 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-04-26T20:01:37.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.229+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 msgr2=0x7fddd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.229+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 0x7fddd006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009560 tx=0x7fdddc008040 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.229+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 msgr2=0x7fddec0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.229+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdde000cb10 tx=0x7fdde000a060 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 shutdown_connections 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddec15fe50 0x7fddec0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddec1598a0 0x7fddec0cb790 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fddd006c8f0 0x7fddd006edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 --2- 172.21.15.88:0/704089270 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fddec06c770 0x7fddec146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 >> 172.21.15.88:0/704089270 conn(0x7fddec0b2d50 msgr2=0x7fddec1615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 shutdown_connections 2024-04-26T20:01:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.230+0000 7fddf4200640 1 -- 172.21.15.88:0/704089270 wait complete. 2024-04-26T20:01:37.251 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:37.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.449+0000 7f8bb1200640 1 Processor -- start 2024-04-26T20:01:37.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.449+0000 7f8bb1200640 1 -- start start 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.450+0000 7f8bb1200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8bac146680 0x7f8bac0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.450+0000 7f8bb1200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8bac0b4b30 0x7f8bac0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8bb1200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bac146a60 con 0x7f8bac0d0e70 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8bb1200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8bac06e970 con 0x7f8bac0b4b30 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8bb1200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8bac0d17f0 con 0x7f8bac146680 2024-04-26T20:01:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8bac0b4b30 0x7f8bac0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8bac0b4b30 0x7f8bac0d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58088/0 (socket says 172.21.15.88:58088) 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 -- 172.21.15.88:0/64387404 learned_addr learned my addr 172.21.15.88:0/64387404 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8bab400640 1 --2- 172.21.15.88:0/64387404 >> v2:172.21.15.132:3300/0 conn(0x7f8bac146680 0x7f8bac0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 -- 172.21.15.88:0/64387404 >> v2:172.21.15.132:3300/0 conn(0x7f8bac146680 msgr2=0x7f8bac0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 --2- 172.21.15.88:0/64387404 >> v2:172.21.15.132:3300/0 conn(0x7f8bac146680 0x7f8bac0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 -- 172.21.15.88:0/64387404 >> v1:172.21.15.132:6789/0 conn(0x7f8bac0d0e70 legacy=0x7f8bac160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.451+0000 7f8babe00640 1 -- 172.21.15.88:0/64387404 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac06dd60 con 0x7f8bac0b4b30 2024-04-26T20:01:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.452+0000 7f8babe00640 1 --2- 172.21.15.88:0/64387404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0b4b30 0x7f8bac0d12b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bac074250 tx=0x7f8ba402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce143101e6f338c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8baa000640 1 -- 172.21.15.88:0/64387404 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4036070 con 0x7f8bac0b4b30 2024-04-26T20:01:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8baa000640 1 -- 172.21.15.88:0/64387404 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba4031600 con 0x7f8bac0b4b30 2024-04-26T20:01:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8baa000640 1 -- 172.21.15.88:0/64387404 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40319d0 con 0x7f8bac0b4b30 2024-04-26T20:01:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 -- 172.21.15.88:0/64387404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0b4b30 msgr2=0x7f8bac0d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 --2- 172.21.15.88:0/64387404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0b4b30 0x7f8bac0d12b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8bac074250 tx=0x7f8ba402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 -- 172.21.15.88:0/64387404 shutdown_connections 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 --2- 172.21.15.88:0/64387404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0b4b30 0x7f8bac0d12b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 --2- 172.21.15.88:0/64387404 >> v2:172.21.15.132:3300/0 conn(0x7f8bac146680 0x7f8bac0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 -- 172.21.15.88:0/64387404 >> 172.21.15.88:0/64387404 conn(0x7f8bac06c740 msgr2=0x7f8bac06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 -- 172.21.15.88:0/64387404 shutdown_connections 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.453+0000 7f8bb1200640 1 -- 172.21.15.88:0/64387404 wait complete. 2024-04-26T20:01:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.455+0000 7f8bb1200640 1 Processor -- start 2024-04-26T20:01:37.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.455+0000 7f8bb1200640 1 -- start start 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bac0b4b30 0x7f8bac15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bac146680 0x7f8bac15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac06e600 con 0x7f8bac0d0e70 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac06e320 con 0x7f8bac146680 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8bb1200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8bac0cd6a0 con 0x7f8bac0b4b30 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58102/0 (socket says 172.21.15.88:58102) 2024-04-26T20:01:37.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.456+0000 7f8baaa00640 1 -- 172.21.15.88:0/2983104864 learned_addr learned my addr 172.21.15.88:0/2983104864 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8bab400640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bac0b4b30 0x7f8bac15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 -- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bac0b4b30 msgr2=0x7f8bac15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bac0b4b30 0x7f8bac15c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 -- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bac146680 msgr2=0x7f8bac15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8babe00640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bac146680 0x7f8bac15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bac146680 0x7f8bac15f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 -- 172.21.15.88:0/2983104864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac13a980 con 0x7f8bac0d0e70 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8baaa00640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c440 tx=0x7f8b9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x7f8bac0d0e70 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980035f0 con 0x7f8bac0d0e70 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac13ac60 con 0x7f8bac0d0e70 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.457+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980039e0 con 0x7f8bac0d0e70 2024-04-26T20:01:37.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.458+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac13b220 con 0x7f8bac0d0e70 2024-04-26T20:01:37.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.459+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac0693c0 con 0x7f8bac0d0e70 2024-04-26T20:01:37.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.460+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b9801a070 con 0x7f8bac0d0e70 2024-04-26T20:01:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.460+0000 7f8ba3400640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 0x7f8b8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.461+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f8b9808aed0 con 0x7f8bac0d0e70 2024-04-26T20:01:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.461+0000 7f8bab400640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 0x7f8b8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.461+0000 7f8bab400640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 0x7f8b8c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c006fd0 tx=0x7f8b9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.465+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b98059330 con 0x7f8bac0d0e70 2024-04-26T20:01:37.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.665+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bac140d70 con 0x7f8bac0d0e70 2024-04-26T20:01:37.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.668+0000 7f8ba3400640 1 -- 172.21.15.88:0/2983104864 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16786 (secure 0 0 0) 0x7f8b980010c0 con 0x7f8bac0d0e70 2024-04-26T20:01:37.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:37.671 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:31.270527+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 msgr2=0x7f8b8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 0x7f8b8c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c006fd0 tx=0x7f8b9c008040 comp rx=0 tx=0).stop 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 msgr2=0x7f8bac13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c440 tx=0x7f8b9800c900 comp rx=0 tx=0).stop 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 shutdown_connections 2024-04-26T20:01:37.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bac146680 0x7f8bac15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bac0d0e70 0x7f8bac13fa30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8b8c06c8f0 0x7f8b8c06edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 --2- 172.21.15.88:0/2983104864 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8bac0b4b30 0x7f8bac15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 >> 172.21.15.88:0/2983104864 conn(0x7f8bac06c740 msgr2=0x7f8bac0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 shutdown_connections 2024-04-26T20:01:37.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.671+0000 7f8bb1200640 1 -- 172.21.15.88:0/2983104864 wait complete. 2024-04-26T20:01:37.691 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:03.091802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:01:04.331500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:37.692 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:37.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.890+0000 7fba82400640 1 Processor -- start 2024-04-26T20:01:37.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.890+0000 7fba82400640 1 -- start start 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba82400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fba7c15d100 0x7fba7c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba82400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0b5ca0 0x7fba7c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba82400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c0af800 con 0x7fba7c0add80 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba82400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0ae340 con 0x7fba7c15d100 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba82400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c0afaa0 con 0x7fba7c0b5ca0 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba7b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fba7c15d100 0x7fba7c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba7b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fba7c15d100 0x7fba7c06c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58106/0 (socket says 172.21.15.88:58106) 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba7b400640 1 -- 172.21.15.88:0/1191641164 learned_addr learned my addr 172.21.15.88:0/1191641164 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.892+0000 7fba7aa00640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0b5ca0 0x7fba7c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.893+0000 7fba7b400640 1 -- 172.21.15.88:0/1191641164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0b5ca0 msgr2=0x7fba7c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:37.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.893+0000 7fba7b400640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0b5ca0 0x7fba7c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.893+0000 7fba7b400640 1 -- 172.21.15.88:0/1191641164 >> v1:172.21.15.88:6789/0 conn(0x7fba7c0add80 legacy=0x7fba7c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.893+0000 7fba7b400640 1 -- 172.21.15.88:0/1191641164 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0afd80 con 0x7fba7c15d100 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.894+0000 7fba7b400640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c06c770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009870 tx=0x7fba6c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7fe4811ae8bbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.894+0000 7fba7a000640 1 -- 172.21.15.88:0/1191641164 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037070 con 0x7fba7c15d100 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.894+0000 7fba7a000640 1 -- 172.21.15.88:0/1191641164 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c032440 con 0x7fba7c15d100 2024-04-26T20:01:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.894+0000 7fba7a000640 1 -- 172.21.15.88:0/1191641164 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c032810 con 0x7fba7c15d100 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 -- 172.21.15.88:0/1191641164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 msgr2=0x7fba7c06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c06c770 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009870 tx=0x7fba6c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 -- 172.21.15.88:0/1191641164 shutdown_connections 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0b5ca0 0x7fba7c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 --2- 172.21.15.88:0/1191641164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c06c770 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 -- 172.21.15.88:0/1191641164 >> 172.21.15.88:0/1191641164 conn(0x7fba7c0b2d50 msgr2=0x7fba7c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 -- 172.21.15.88:0/1191641164 shutdown_connections 2024-04-26T20:01:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.895+0000 7fba82400640 1 -- 172.21.15.88:0/1191641164 wait complete. 2024-04-26T20:01:37.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.896+0000 7fba82400640 1 Processor -- start 2024-04-26T20:01:37.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.897+0000 7fba82400640 1 -- start start 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0add80 0x7fba7c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 0x7fba7c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c09ee50 con 0x7fba7c15d100 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0b0340 con 0x7fba7c0b5ca0 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba82400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c160b70 con 0x7fba7c0add80 2024-04-26T20:01:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58120/0 (socket says 172.21.15.88:58120) 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 -- 172.21.15.88:0/3956127109 learned_addr learned my addr 172.21.15.88:0/3956127109 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7aa00640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 0x7fba7c06f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 -- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0add80 msgr2=0x7fba7c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0add80 0x7fba7c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 -- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 msgr2=0x7fba7c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 0x7fba7c06f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7be00640 1 -- 172.21.15.88:0/3956127109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c1f5f60 con 0x7fba7c15d100 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.898+0000 7fba7aa00640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 0x7fba7c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba7be00640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c6b0 tx=0x7fba7400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74003b70 con 0x7fba7c15d100 2024-04-26T20:01:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba74003d30 con 0x7fba7c15d100 2024-04-26T20:01:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74005310 con 0x7fba7c15d100 2024-04-26T20:01:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c1f6240 con 0x7fba7c15d100 2024-04-26T20:01:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.899+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1f6c30 con 0x7fba7c15d100 2024-04-26T20:01:37.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.900+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c069650 con 0x7fba7c15d100 2024-04-26T20:01:37.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.901+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba740054b0 con 0x7fba7c15d100 2024-04-26T20:01:37.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.902+0000 7fba73400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 0x7fba5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:37.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.903+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fba7408ba20 con 0x7fba7c15d100 2024-04-26T20:01:37.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.903+0000 7fba7b400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 0x7fba5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.903+0000 7fba7b400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 0x7fba5c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba6c034000 tx=0x7fba6c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:37.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:37.907+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba74059de0 con 0x7fba7c15d100 2024-04-26T20:01:38.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.113+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba7c073d00 con 0x7fba7c15d100 2024-04-26T20:01:38.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.114+0000 7fba73400640 1 -- 172.21.15.88:0/3956127109 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fba740596e0 con 0x7fba7c15d100 2024-04-26T20:01:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:38.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 msgr2=0x7fba5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 0x7fba5c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fba6c034000 tx=0x7fba6c02c000 comp rx=0 tx=0).stop 2024-04-26T20:01:38.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 msgr2=0x7fba7c1f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fba7400c6b0 tx=0x7fba7400cb70 comp rx=0 tx=0).stop 2024-04-26T20:01:38.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 shutdown_connections 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c15d100 0x7fba7c1f5270 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c0b5ca0 0x7fba7c06f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fba5c06c8f0 0x7fba5c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 --2- 172.21.15.88:0/3956127109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fba7c0add80 0x7fba7c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 >> 172.21.15.88:0/3956127109 conn(0x7fba7c0b2d50 msgr2=0x7fba7c0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.116+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 shutdown_connections 2024-04-26T20:01:38.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.117+0000 7fba82400640 1 -- 172.21.15.88:0/3956127109 wait complete. 2024-04-26T20:01:38.137 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:01:38.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.338+0000 7fb715000640 1 Processor -- start 2024-04-26T20:01:38.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.338+0000 7fb715000640 1 -- start start 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.339+0000 7fb715000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb710160d60 0x7fb7100b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.339+0000 7fb715000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7100b4d20 con 0x7fb710161140 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.339+0000 7fb715000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7100b5000 con 0x7fb710146280 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.339+0000 7fb715000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb71006e5f0 con 0x7fb710160d60 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70ea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb710161140 0x7fb71006dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55602/0 (socket says 172.21.15.88:55602) 2024-04-26T20:01:38.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70ea00640 1 -- 172.21.15.88:0/363326300 learned_addr learned my addr 172.21.15.88:0/363326300 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:38.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70f400640 1 --2- 172.21.15.88:0/363326300 >> v2:172.21.15.132:3300/0 conn(0x7fb710160d60 0x7fb7100b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238769511 0 0) 0x7fb7100b4d20 con 0x7fb710161140 2024-04-26T20:01:38.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f4003520 con 0x7fb710161140 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1020959436 0 0) 0x7fb6f4003520 con 0x7fb710161140 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 >> v2:172.21.15.132:3300/0 conn(0x7fb710160d60 msgr2=0x7fb7100b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 --2- 172.21.15.88:0/363326300 >> v2:172.21.15.132:3300/0 conn(0x7fb710160d60 0x7fb7100b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 >> v1:172.21.15.132:6789/0 conn(0x7fb710146280 legacy=0x7fb7100cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7100b52e0 con 0x7fb710161140 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.340+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb6fc001670 con 0x7fb710161140 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 >> v1:172.21.15.88:6789/0 conn(0x7fb710161140 legacy=0x7fb71006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f4003a20 0x7fb6f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f4004800 0x7fb6f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb6f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb71006e5f0 con 0x7fb6f40051c0 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f400c160 con 0x7fb6f4004800 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f400c360 con 0x7fb6f4003a20 2024-04-26T20:01:38.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb6f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70ea00640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f4004800 0x7fb6f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70f400640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f4003a20 0x7fb6f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 -- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f4003a20 msgr2=0x7fb6f4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f4003a20 0x7fb6f4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 -- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f4004800 msgr2=0x7fb6f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f4004800 0x7fb6f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70fe00640 1 -- 172.21.15.88:0/363326300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f4003520 con 0x7fb6f40051c0 2024-04-26T20:01:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.341+0000 7fb70f400640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f4003a20 0x7fb6f4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.342+0000 7fb70fe00640 1 -- 172.21.15.88:0/363326300 reap_dead start 2024-04-26T20:01:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.342+0000 7fb70fe00640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb6f400b9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb708001e50 tx=0x7fb6f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e2c19008b3845 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708001a70 con 0x7fb6f40051c0 2024-04-26T20:01:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708003970 con 0x7fb6f40051c0 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb70e000640 1 -- 172.21.15.88:0/363326300 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708003d60 con 0x7fb6f40051c0 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 -- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 msgr2=0x7fb6f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb6f400b9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb708001e50 tx=0x7fb6f4005830 comp rx=0 tx=0).stop 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 -- 172.21.15.88:0/363326300 shutdown_connections 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 --2- 172.21.15.88:0/363326300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb6f400b9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 -- 172.21.15.88:0/363326300 >> 172.21.15.88:0/363326300 conn(0x7fb7100b2900 msgr2=0x7fb7100b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 -- 172.21.15.88:0/363326300 shutdown_connections 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.343+0000 7fb715000640 1 -- 172.21.15.88:0/363326300 wait complete. 2024-04-26T20:01:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.345+0000 7fb715000640 1 Processor -- start 2024-04-26T20:01:38.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.345+0000 7fb715000640 1 -- start start 2024-04-26T20:01:38.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb71013eb60 0x7fb7101412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7101f2120 0x7fb7101f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb710161770 con 0x7fb6f40051c0 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100cf6f0 con 0x7fb7101f2120 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb715000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb7101618f0 con 0x7fb71013eb60 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb70f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb70f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58130/0 (socket says 172.21.15.88:58130) 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.346+0000 7fb70f400640 1 -- 172.21.15.88:0/3870866429 learned_addr learned my addr 172.21.15.88:0/3870866429 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 -- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb71013eb60 msgr2=0x7fb7101412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb71013eb60 0x7fb7101412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 -- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7101f2120 msgr2=0x7fb7101f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7101f2120 0x7fb7101f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 -- 172.21.15.88:0/3870866429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71015b230 con 0x7fb6f40051c0 2024-04-26T20:01:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb70f400640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb700010910 tx=0x7fb700010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f040 con 0x7fb6f40051c0 2024-04-26T20:01:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70000ec20 con 0x7fb6f40051c0 2024-04-26T20:01:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70001c400 con 0x7fb6f40051c0 2024-04-26T20:01:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.347+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71015b510 con 0x7fb6f40051c0 2024-04-26T20:01:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.348+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71015e230 con 0x7fb6f40051c0 2024-04-26T20:01:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.349+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7100693c0 con 0x7fb6f40051c0 2024-04-26T20:01:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.350+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb70001c5a0 con 0x7fb6f40051c0 2024-04-26T20:01:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.350+0000 7fb707400640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.351+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fb70008c220 con 0x7fb6f40051c0 2024-04-26T20:01:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.351+0000 7fb70ea00640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.351+0000 7fb70ea00640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 0x7fb6f006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc001eb0 tx=0x7fb6fc001870 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.355+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb700018030 con 0x7fb6f40051c0 2024-04-26T20:01:38.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.553+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb710143a00 con 0x7fb6f40051c0 2024-04-26T20:01:38.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.553+0000 7fb707400640 1 -- 172.21.15.88:0/3870866429 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7fb700059f90 con 0x7fb6f40051c0 2024-04-26T20:01:38.555 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 msgr2=0x7fb6f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 0x7fb6f006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc001eb0 tx=0x7fb6fc001870 comp rx=0 tx=0).stop 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 msgr2=0x7fb7101f6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb700010910 tx=0x7fb700010dd0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 shutdown_connections 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7101f2120 0x7fb7101f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb71013eb60 0x7fb7101412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 --2- 172.21.15.88:0/3870866429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f40051c0 0x7fb7101f6980 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 >> 172.21.15.88:0/3870866429 conn(0x7fb7100b2900 msgr2=0x7fb71015a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:38.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 shutdown_connections 2024-04-26T20:01:38.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.557+0000 7fb715000640 1 -- 172.21.15.88:0/3870866429 wait complete. 2024-04-26T20:01:38.576 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:01:38.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.772+0000 7f8f3be00640 1 Processor -- start 2024-04-26T20:01:38.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.772+0000 7f8f3be00640 1 -- start start 2024-04-26T20:01:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.773+0000 7f8f3be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f3c156370 0x7f8f3c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.773+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 0x7f8f3c0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.773+0000 7f8f3be00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3c156750 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3c15d430 con 0x7f8f3c156370 2024-04-26T20:01:38.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c0d16d0 con 0x7f8f3c15c760 2024-04-26T20:01:38.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8f3c0d12a0 0x7f8f3c15f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:55612/0 (socket says 172.21.15.88:55612) 2024-04-26T20:01:38.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3b400640 1 -- 172.21.15.88:0/1479829964 learned_addr learned my addr 172.21.15.88:0/1479829964 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/1479829964 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c156370 0x7f8f3c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f3a000640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 0x7f8f3c0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31504531 0 0) 0x7f8f34001490 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.774+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f20003520 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3519951641 0 0) 0x7f8f20003520 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 msgr2=0x7f8f3c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 0x7f8f3c0d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c156370 msgr2=0x7f8f3c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 --2- 172.21.15.88:0/1479829964 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c156370 0x7f8f3c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c15cd60 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f8f34001bc0 con 0x7f8f3c0d12a0 2024-04-26T20:01:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 >> v1:172.21.15.88:6789/0 conn(0x7f8f3c0d12a0 legacy=0x7f8f3c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 0x7f8f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c15d430 con 0x7f8f200039b0 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0d16d0 con 0x7f8f20004800 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2000dc50 con 0x7f8f3c15c760 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.775+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3a000640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3aa00640 1 -- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 msgr2=0x7f8f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15c760 0x7f8f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3aa00640 1 -- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 msgr2=0x7f8f20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.776+0000 7f8f3aa00640 1 -- 172.21.15.88:0/1479829964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20003520 con 0x7f8f200039b0 2024-04-26T20:01:38.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba30 tx=0x7f8f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b814ed3eeb769b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c030070 con 0x7f8f200039b0 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c02ae10 con 0x7f8f200039b0 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f39600640 1 -- 172.21.15.88:0/1479829964 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00b600 con 0x7f8f200039b0 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f3be00640 1 -- 172.21.15.88:0/1479829964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 msgr2=0x7f8f20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.777+0000 7f8f3be00640 1 --2- 172.21.15.88:0/1479829964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba30 tx=0x7f8f20005830 comp rx=0 tx=0).stop 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.778+0000 7f8f3b400640 1 -- 172.21.15.88:0/1479829964 reap_dead start 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.778+0000 7f8f3be00640 1 -- 172.21.15.88:0/1479829964 shutdown_connections 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.778+0000 7f8f3be00640 1 -- 172.21.15.88:0/1479829964 >> 172.21.15.88:0/1479829964 conn(0x7f8f3c06c530 msgr2=0x7f8f3c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.778+0000 7f8f3be00640 1 -- 172.21.15.88:0/1479829964 shutdown_connections 2024-04-26T20:01:38.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.778+0000 7f8f3be00640 1 -- 172.21.15.88:0/1479829964 wait complete. 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.779+0000 7f8f3be00640 1 Processor -- start 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 -- start start 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 0x7f8f3c1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c072600 0x7f8f3c072a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c15d960 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c15d680 con 0x7f8f3c072600 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.780+0000 7f8f3be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b3cb0 con 0x7f8f200039b0 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58152/0 (socket says 172.21.15.88:58152) 2024-04-26T20:01:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 -- 172.21.15.88:0/2546982613 learned_addr learned my addr 172.21.15.88:0/2546982613 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 0x7f8f3c1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3a000640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c072600 0x7f8f3c072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 -- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 msgr2=0x7f8f3c1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 0x7f8f3c1507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 -- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c072600 msgr2=0x7f8f3c072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c072600 0x7f8f3c072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 -- 172.21.15.88:0/2546982613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c0afde0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 0x7f8f3c1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.781+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f34007fd0 tx=0x7f8f3400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.782+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400a5e0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.782+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3400a780 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.782+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ab50 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.782+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c0b00c0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.782+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1f51f0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.783+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c072fc0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.784+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f3400acf0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.785+0000 7f8f32a00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 0x7f8f1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:38.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.785+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f8f3408b170 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.786+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 0x7f8f1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:38.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.786+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 0x7f8f1c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba00 tx=0x7f8f2c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:38.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.790+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f3401a030 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.989+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8f3c0693c0 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.990+0000 7f8f32a00640 1 -- 172.21.15.88:0/2546982613 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f8f3c156750 con 0x7f8f3c0af1e0 2024-04-26T20:01:38.991 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:01:38.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.992+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 msgr2=0x7f8f1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.992+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 0x7f8f1c06ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba00 tx=0x7f8f2c030000 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.992+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 msgr2=0x7f8f3c0af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.992+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8f34007fd0 tx=0x7f8f3400ccd0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 shutdown_connections 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f3c0af1e0 0x7f8f3c0af660 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c072600 0x7f8f3c072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1c06c920 0x7f8f1c06ede0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2546982613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f200039b0 0x7f8f3c1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 >> 172.21.15.88:0/2546982613 conn(0x7f8f3c06c530 msgr2=0x7f8f3c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 shutdown_connections 2024-04-26T20:01:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:38.993+0000 7f8f3be00640 1 -- 172.21.15.88:0/2546982613 wait complete. 2024-04-26T20:01:39.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:39.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.213+0000 7f3816a00640 1 Processor -- start 2024-04-26T20:01:39.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.213+0000 7f3816a00640 1 -- start start 2024-04-26T20:01:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3816a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f381015cdf0 0x7f38100add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3816a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f381006ce80 0x7f38100b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3816a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38100af9a0 con 0x7f38100cd650 2024-04-26T20:01:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3816a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f38100ae2c0 con 0x7f381006ce80 2024-04-26T20:01:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3816a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f38100b05b0 con 0x7f381015cdf0 2024-04-26T20:01:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3807400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f38100cd650 0x7f381006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41480/0 (socket says 172.21.15.88:41480) 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3807400640 1 -- 172.21.15.88:0/1750105697 learned_addr learned my addr 172.21.15.88:0/1750105697 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3814a00640 1 --2- 172.21.15.88:0/1750105697 >> v2:172.21.15.88:3300/0 conn(0x7f381006ce80 0x7f38100b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f380fe00640 1 --2- 172.21.15.88:0/1750105697 >> v2:172.21.15.132:3300/0 conn(0x7f381015cdf0 0x7f38100add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3814a00640 1 -- 172.21.15.88:0/1750105697 >> v2:172.21.15.132:3300/0 conn(0x7f381015cdf0 msgr2=0x7f38100add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3814a00640 1 --2- 172.21.15.88:0/1750105697 >> v2:172.21.15.132:3300/0 conn(0x7f381015cdf0 0x7f38100add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3814a00640 1 -- 172.21.15.88:0/1750105697 >> v1:172.21.15.88:6789/0 conn(0x7f38100cd650 legacy=0x7f381006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.215+0000 7f3814a00640 1 -- 172.21.15.88:0/1750105697 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38100afc80 con 0x7f381006ce80 2024-04-26T20:01:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.216+0000 7f3814a00640 1 --2- 172.21.15.88:0/1750105697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f38100b3d10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f38100b3950 tx=0x7f380802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabab4562443834c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.216+0000 7f380f400640 1 -- 172.21.15.88:0/1750105697 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808037070 con 0x7f381006ce80 2024-04-26T20:01:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.216+0000 7f380f400640 1 -- 172.21.15.88:0/1750105697 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3808032440 con 0x7f381006ce80 2024-04-26T20:01:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.216+0000 7f380f400640 1 -- 172.21.15.88:0/1750105697 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808032810 con 0x7f381006ce80 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 -- 172.21.15.88:0/1750105697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 msgr2=0x7f38100b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 --2- 172.21.15.88:0/1750105697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f38100b3d10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f38100b3950 tx=0x7f380802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 -- 172.21.15.88:0/1750105697 shutdown_connections 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 --2- 172.21.15.88:0/1750105697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f38100b3d10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 --2- 172.21.15.88:0/1750105697 >> v2:172.21.15.132:3300/0 conn(0x7f381015cdf0 0x7f38100add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 -- 172.21.15.88:0/1750105697 >> 172.21.15.88:0/1750105697 conn(0x7f38100b2d50 msgr2=0x7f38100b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 -- 172.21.15.88:0/1750105697 shutdown_connections 2024-04-26T20:01:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.217+0000 7f3816a00640 1 -- 172.21.15.88:0/1750105697 wait complete. 2024-04-26T20:01:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.219+0000 7f3816a00640 1 Processor -- start 2024-04-26T20:01:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.219+0000 7f3816a00640 1 -- start start 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38100cd650 0x7f3810147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f381015cdf0 0x7f3810153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f381015d1d0 con 0x7f381006ce80 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f38100b0240 con 0x7f381015cdf0 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f3816a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f381015d350 con 0x7f38100cd650 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f380fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f380fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46112/0 (socket says 172.21.15.88:46112) 2024-04-26T20:01:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.220+0000 7f380fe00640 1 -- 172.21.15.88:0/3382466610 learned_addr learned my addr 172.21.15.88:0/3382466610 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 -- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38100cd650 msgr2=0x7f3810147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f3807400640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38100cd650 0x7f3810147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f3814a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f381015cdf0 0x7f3810153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38100cd650 0x7f3810147bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 -- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f381015cdf0 msgr2=0x7f3810153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f381015cdf0 0x7f3810153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 -- 172.21.15.88:0/3382466610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3810148a60 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380fe00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f380000d800 tx=0x7f380000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000b6c0 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800019070 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101481c0 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000ba80 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.221+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3810148780 con 0x7f381006ce80 2024-04-26T20:01:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.222+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3810069540 con 0x7f381006ce80 2024-04-26T20:01:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.223+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f380000bc20 con 0x7f381006ce80 2024-04-26T20:01:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.224+0000 7f380cc00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 0x7f37ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.224+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f380008bfb0 con 0x7f381006ce80 2024-04-26T20:01:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.225+0000 7f3807400640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 0x7f37ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.225+0000 7f3807400640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 0x7f37ec06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f381014e1e0 tx=0x7f37f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.230+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3810148a60 con 0x7f381006ce80 2024-04-26T20:01:39.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.466+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38100b5af0 con 0x7f381006ce80 2024-04-26T20:01:39.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.468+0000 7f380cc00640 1 -- 172.21.15.88:0/3382466610 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7f380005a410 con 0x7f381006ce80 2024-04-26T20:01:39.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:39.470 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/809249379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":809249379},{"type":"v1","addr":"172.21.15.88:6837","nonce":809249379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:39.470 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 msgr2=0x7f37ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 0x7f37ec06ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f381014e1e0 tx=0x7f37f8009040 comp rx=0 tx=0).stop 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 msgr2=0x7f3810147690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f380000d800 tx=0x7f380000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 shutdown_connections 2024-04-26T20:01:39.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f381015cdf0 0x7f3810153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f37ec06c920 0x7f37ec06ede0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38100cd650 0x7f3810147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 --2- 172.21.15.88:0/3382466610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f381006ce80 0x7f3810147690 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 >> 172.21.15.88:0/3382466610 conn(0x7f38100b2d50 msgr2=0x7f381006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 shutdown_connections 2024-04-26T20:01:39.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.471+0000 7f3816a00640 1 -- 172.21.15.88:0/3382466610 wait complete. 2024-04-26T20:01:39.490 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:39.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 Processor -- start 2024-04-26T20:01:39.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 -- start start 2024-04-26T20:01:39.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbc6006c770 0x7fbc6006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc60073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6006d090 con 0x7fbc6015cfd0 2024-04-26T20:01:39.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fbc600af460 con 0x7fbc6006c770 2024-04-26T20:01:39.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.688+0000 7fbc67a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbc600ae850 con 0x7fbc6015cb70 2024-04-26T20:01:39.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.689+0000 7fbc65000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbc6006c770 0x7fbc6006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.689+0000 7fbc5fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc60073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.689+0000 7fbc65000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fbc6006c770 0x7fbc6006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:37316/0 (socket says 172.21.15.88:37316) 2024-04-26T20:01:39.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.689+0000 7fbc65a00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fbc6015cfd0 0x7fbc60073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:44128/0 (socket says 172.21.15.88:44128) 2024-04-26T20:01:39.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.689+0000 7fbc65000640 1 -- 172.21.15.88:0/61005523 learned_addr learned my addr 172.21.15.88:0/61005523 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:39.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.690+0000 7fbc65000640 1 -- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 msgr2=0x7fbc60073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:39.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.690+0000 7fbc65000640 1 --2- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc60073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.690+0000 7fbc65000640 1 -- 172.21.15.88:0/61005523 >> v1:172.21.15.132:6789/0 conn(0x7fbc6015cfd0 legacy=0x7fbc60073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.690+0000 7fbc65000640 1 -- 172.21.15.88:0/61005523 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc600aeb30 con 0x7fbc6006c770 2024-04-26T20:01:39.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.692+0000 7fbc65000640 1 --2- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc6006cb50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009870 tx=0x7fbc5402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324e00b2f4e883a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.692+0000 7fbc5f400640 1 -- 172.21.15.88:0/61005523 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54037070 con 0x7fbc6006c770 2024-04-26T20:01:39.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc5f400640 1 -- 172.21.15.88:0/61005523 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54032440 con 0x7fbc6006c770 2024-04-26T20:01:39.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc5f400640 1 -- 172.21.15.88:0/61005523 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54032810 con 0x7fbc6006c770 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 -- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 msgr2=0x7fbc6006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 --2- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc6006cb50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009870 tx=0x7fbc5402a250 comp rx=0 tx=0).stop 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 -- 172.21.15.88:0/61005523 shutdown_connections 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 --2- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc60073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 --2- 172.21.15.88:0/61005523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc6006cb50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 -- 172.21.15.88:0/61005523 >> 172.21.15.88:0/61005523 conn(0x7fbc600b2d50 msgr2=0x7fbc600b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:39.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 -- 172.21.15.88:0/61005523 shutdown_connections 2024-04-26T20:01:39.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.693+0000 7fbc67a00640 1 -- 172.21.15.88:0/61005523 wait complete. 2024-04-26T20:01:39.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.695+0000 7fbc67a00640 1 Processor -- start 2024-04-26T20:01:39.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.695+0000 7fbc67a00640 1 -- start start 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc60150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc6006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6015c600 con 0x7fbc6015cfd0 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fbc600af0f0 con 0x7fbc6006c770 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc67a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6015c780 con 0x7fbc6015cb70 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.696+0000 7fbc65a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46128/0 (socket says 172.21.15.88:46128) 2024-04-26T20:01:39.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 -- 172.21.15.88:0/626546314 learned_addr learned my addr 172.21.15.88:0/626546314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc5fe00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc6006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65000640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc60150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 -- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 msgr2=0x7fbc6006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc6006ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 -- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 msgr2=0x7fbc60150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc60150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 -- 172.21.15.88:0/626546314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6015b830 con 0x7fbc6015cfd0 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.697+0000 7fbc65a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c730 tx=0x7fbc5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.698+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58017070 con 0x7fbc6015cfd0 2024-04-26T20:01:39.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.698+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5800ce40 con 0x7fbc6015cfd0 2024-04-26T20:01:39.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.698+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6015bb10 con 0x7fbc6015cfd0 2024-04-26T20:01:39.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.698+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800a840 con 0x7fbc6015cfd0 2024-04-26T20:01:39.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.698+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6014fbf0 con 0x7fbc6015cfd0 2024-04-26T20:01:39.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.699+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6014fdf0 con 0x7fbc6015cfd0 2024-04-26T20:01:39.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.700+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc5801a070 con 0x7fbc6015cfd0 2024-04-26T20:01:39.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.701+0000 7fbc5cc00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 0x7fbc4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:39.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.701+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fbc5808b110 con 0x7fbc6015cfd0 2024-04-26T20:01:39.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.702+0000 7fbc65000640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 0x7fbc4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:39.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.702+0000 7fbc65000640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 0x7fbc4006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034000 tx=0x7fbc5402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:39.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.706+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc6014fdf0 con 0x7fbc6015cfd0 2024-04-26T20:01:39.786 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:01:39.793 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:01:39.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.909+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc6006ec40 con 0x7fbc6015cfd0 2024-04-26T20:01:39.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.912+0000 7fbc5cc00640 1 -- 172.21.15.88:0/626546314 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16786 (secure 0 0 0) 0x7fbc58058df0 con 0x7fbc6015cfd0 2024-04-26T20:01:39.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:39.914 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:31.270527+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 msgr2=0x7fbc4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 0x7fbc4006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034000 tx=0x7fbc5402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 msgr2=0x7fbc6006f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c730 tx=0x7fbc5800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 shutdown_connections 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc6015cfd0 0x7fbc6006f3a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fbc4006c8f0 0x7fbc4006edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fbc6015cb70 0x7fbc6006ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 --2- 172.21.15.88:0/626546314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc6006c770 0x7fbc60150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 >> 172.21.15.88:0/626546314 conn(0x7fbc600b2d50 msgr2=0x7fbc6015d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 shutdown_connections 2024-04-26T20:01:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:39.915+0000 7fbc67a00640 1 -- 172.21.15.88:0/626546314 wait complete. 2024-04-26T20:01:39.935 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:01:39.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:40.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.179+0000 7fdaa8a00640 1 Processor -- start 2024-04-26T20:01:40.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.179+0000 7fdaa8a00640 1 -- start start 2024-04-26T20:01:40.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.179+0000 7fdaa8a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdaa406ce80 0x7fdaa40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa8a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdaa40cd650 0x7fdaa40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa8a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa40af9a0 con 0x7fdaa415cdf0 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa8a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa40ae2c0 con 0x7fdaa40cd650 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa8a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa40b05b0 con 0x7fdaa406ce80 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa1e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdaa415cdf0 0x7fdaa406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41486/0 (socket says 172.21.15.88:41486) 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa1e00640 1 -- 172.21.15.88:0/1301943404 learned_addr learned my addr 172.21.15.88:0/1301943404 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa2800640 1 --2- 172.21.15.88:0/1301943404 >> v2:172.21.15.88:3300/0 conn(0x7fdaa40cd650 0x7fdaa40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa1400640 1 -- 172.21.15.88:0/1301943404 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010940143 0 0) 0x7fdaa40af9a0 con 0x7fdaa415cdf0 2024-04-26T20:01:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa1400640 1 -- 172.21.15.88:0/1301943404 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda88003520 con 0x7fdaa415cdf0 2024-04-26T20:01:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fda99400640 1 --2- 172.21.15.88:0/1301943404 >> v2:172.21.15.132:3300/0 conn(0x7fdaa406ce80 0x7fdaa40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.180+0000 7fdaa2800640 1 -- 172.21.15.88:0/1301943404 >> v2:172.21.15.132:3300/0 conn(0x7fdaa406ce80 msgr2=0x7fdaa40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.181+0000 7fdaa2800640 1 --2- 172.21.15.88:0/1301943404 >> v2:172.21.15.132:3300/0 conn(0x7fdaa406ce80 0x7fdaa40add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.181+0000 7fdaa2800640 1 -- 172.21.15.88:0/1301943404 >> v1:172.21.15.88:6789/0 conn(0x7fdaa415cdf0 legacy=0x7fdaa406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.181+0000 7fdaa2800640 1 -- 172.21.15.88:0/1301943404 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40afc80 con 0x7fdaa40cd650 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa2800640 1 --2- 172.21.15.88:0/1301943404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa40b3d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40d1d00 tx=0x7fda9c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18beb1a13c055838 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa1400640 1 -- 172.21.15.88:0/1301943404 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c007740 con 0x7fdaa40cd650 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa1400640 1 -- 172.21.15.88:0/1301943404 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c0078e0 con 0x7fdaa40cd650 2024-04-26T20:01:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa1400640 1 -- 172.21.15.88:0/1301943404 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c007c90 con 0x7fdaa40cd650 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 -- 172.21.15.88:0/1301943404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 msgr2=0x7fdaa40b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/1301943404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa40b3d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40d1d00 tx=0x7fda9c02a680 comp rx=0 tx=0).stop 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 -- 172.21.15.88:0/1301943404 shutdown_connections 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/1301943404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa40b3d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/1301943404 >> v2:172.21.15.132:3300/0 conn(0x7fdaa406ce80 0x7fdaa40add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.182+0000 7fdaa8a00640 1 -- 172.21.15.88:0/1301943404 >> 172.21.15.88:0/1301943404 conn(0x7fdaa40b2d50 msgr2=0x7fdaa40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.183+0000 7fdaa8a00640 1 -- 172.21.15.88:0/1301943404 shutdown_connections 2024-04-26T20:01:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.183+0000 7fdaa8a00640 1 -- 172.21.15.88:0/1301943404 wait complete. 2024-04-26T20:01:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.184+0000 7fdaa8a00640 1 Processor -- start 2024-04-26T20:01:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.184+0000 7fdaa8a00640 1 -- start start 2024-04-26T20:01:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa406ce80 0x7fdaa4149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaa415cdf0 0x7fdaa40cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa415d1d0 con 0x7fdaa40cd650 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40b0240 con 0x7fdaa406ce80 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fdaa8a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa415d350 con 0x7fdaa415cdf0 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.185+0000 7fda99400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46154/0 (socket says 172.21.15.88:46154) 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 -- 172.21.15.88:0/337873109 learned_addr learned my addr 172.21.15.88:0/337873109 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 -- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaa415cdf0 msgr2=0x7fdaa40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaa415cdf0 0x7fdaa40cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 -- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa406ce80 msgr2=0x7fdaa4149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa406ce80 0x7fdaa4149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 -- 172.21.15.88:0/337873109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40d00e0 con 0x7fdaa40cd650 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda99400640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d830 tx=0x7fda8c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.186+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00b6c0 con 0x7fdaa40cd650 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.187+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda8c019070 con 0x7fdaa40cd650 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.187+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa4149d40 con 0x7fdaa40cd650 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.187+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00ba80 con 0x7fdaa40cd650 2024-04-26T20:01:40.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.187+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41f3280 con 0x7fdaa40cd650 2024-04-26T20:01:40.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.188+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4069540 con 0x7fdaa40cd650 2024-04-26T20:01:40.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.189+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda8c01e460 con 0x7fdaa40cd650 2024-04-26T20:01:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.190+0000 7fda9aa00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.190+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fda8c08c290 con 0x7fdaa40cd650 2024-04-26T20:01:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.190+0000 7fdaa1e00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.191+0000 7fdaa1e00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 0x7fda8006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fda94000f60 tx=0x7fda940078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.194+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdaa4149d40 con 0x7fdaa40cd650 2024-04-26T20:01:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.434+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa4069e70 con 0x7fdaa40cd650 2024-04-26T20:01:40.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.437+0000 7fda9aa00640 1 -- 172.21.15.88:0/337873109 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7fda8c05a660 con 0x7fdaa40cd650 2024-04-26T20:01:40.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:40.438 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/809249379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":809249379},{"type":"v1","addr":"172.21.15.88:6837","nonce":809249379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:40.438 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 msgr2=0x7fda8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 0x7fda8006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fda94000f60 tx=0x7fda940078f0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 msgr2=0x7fdaa415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d830 tx=0x7fda8c00dcf0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 shutdown_connections 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda8006c8f0 0x7fda8006edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdaa415cdf0 0x7fdaa40cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdaa40cd650 0x7fdaa415d530 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 --2- 172.21.15.88:0/337873109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa406ce80 0x7fdaa4149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 >> 172.21.15.88:0/337873109 conn(0x7fdaa40b2d50 msgr2=0x7fdaa406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:40.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 shutdown_connections 2024-04-26T20:01:40.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.440+0000 7fdaa8a00640 1 -- 172.21.15.88:0/337873109 wait complete. 2024-04-26T20:01:40.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:40.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.658+0000 7efdb8e00640 1 Processor -- start 2024-04-26T20:01:40.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.658+0000 7efdb8e00640 1 -- start start 2024-04-26T20:01:40.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.660+0000 7efdb8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb40add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.660+0000 7efdb8e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 0x7efdb40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.660+0000 7efdb8e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb40af9a0 con 0x7efdb40cd650 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.660+0000 7efdb8e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7efdb40ae2c0 con 0x7efdb406ce80 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.660+0000 7efdb8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efdb40b05b0 con 0x7efdb415cdf0 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.661+0000 7efdb2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 0x7efdb40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.661+0000 7efdb1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.661+0000 7efdb2800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 0x7efdb40b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:37344/0 (socket says 172.21.15.88:37344) 2024-04-26T20:01:40.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.661+0000 7efdb2800640 1 -- 172.21.15.88:0/1361799109 learned_addr learned my addr 172.21.15.88:0/1361799109 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.661+0000 7efdb1400640 1 -- 172.21.15.88:0/1361799109 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830716679 0 0) 0x7efdb40af9a0 con 0x7efdb40cd650 2024-04-26T20:01:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.662+0000 7efdb1400640 1 -- 172.21.15.88:0/1361799109 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd98003520 con 0x7efdb40cd650 2024-04-26T20:01:40.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.662+0000 7efdb1e00640 1 -- 172.21.15.88:0/1361799109 >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 msgr2=0x7efdb40b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.662+0000 7efdb1e00640 1 --2- 172.21.15.88:0/1361799109 >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 0x7efdb40b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.662+0000 7efdb1e00640 1 -- 172.21.15.88:0/1361799109 >> v1:172.21.15.132:6789/0 conn(0x7efdb40cd650 legacy=0x7efdb406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.662+0000 7efdb1e00640 1 -- 172.21.15.88:0/1361799109 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb40afc80 con 0x7efdb415cdf0 2024-04-26T20:01:40.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.663+0000 7efdb1e00640 1 --2- 172.21.15.88:0/1361799109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb40add80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2f1c269f0679ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.663+0000 7efdb1400640 1 -- 172.21.15.88:0/1361799109 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4037070 con 0x7efdb415cdf0 2024-04-26T20:01:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.663+0000 7efdb1400640 1 -- 172.21.15.88:0/1361799109 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda4032440 con 0x7efdb415cdf0 2024-04-26T20:01:40.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.663+0000 7efdb1400640 1 -- 172.21.15.88:0/1361799109 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4032810 con 0x7efdb415cdf0 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 -- 172.21.15.88:0/1361799109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 msgr2=0x7efdb40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 --2- 172.21.15.88:0/1361799109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb40add80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efda40097a0 tx=0x7efda402a180 comp rx=0 tx=0).stop 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 -- 172.21.15.88:0/1361799109 shutdown_connections 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 --2- 172.21.15.88:0/1361799109 >> v2:172.21.15.132:3300/0 conn(0x7efdb406ce80 0x7efdb40b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 --2- 172.21.15.88:0/1361799109 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb40add80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 -- 172.21.15.88:0/1361799109 >> 172.21.15.88:0/1361799109 conn(0x7efdb40b2d50 msgr2=0x7efdb40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 -- 172.21.15.88:0/1361799109 shutdown_connections 2024-04-26T20:01:40.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.664+0000 7efdb8e00640 1 -- 172.21.15.88:0/1361799109 wait complete. 2024-04-26T20:01:40.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.667+0000 7efdb8e00640 1 Processor -- start 2024-04-26T20:01:40.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.667+0000 7efdb8e00640 1 -- start start 2024-04-26T20:01:40.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdb40cd650 0x7efdb415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efdb415d1d0 con 0x7efdb406ce80 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efdb40b0240 con 0x7efdb40cd650 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb8e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efdb415d350 con 0x7efdb415cdf0 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46168/0 (socket says 172.21.15.88:46168) 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.668+0000 7efdb1e00640 1 -- 172.21.15.88:0/497393553 learned_addr learned my addr 172.21.15.88:0/497393553 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efda9400640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdb40cd650 0x7efdb415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb2800640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 -- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 msgr2=0x7efdb415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb415d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 -- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdb40cd650 msgr2=0x7efdb415fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdb40cd650 0x7efdb415fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 -- 172.21.15.88:0/497393553 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb41545d0 con 0x7efdb406ce80 2024-04-26T20:01:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.669+0000 7efdb1e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efda4000c00 tx=0x7efda40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.670+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4041020 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.670+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efda403d640 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.670+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4037040 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.670+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb41548b0 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.670+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb4154e70 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.671+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb4069540 con 0x7efdb406ce80 2024-04-26T20:01:40.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.672+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efda4044020 con 0x7efdb406ce80 2024-04-26T20:01:40.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.673+0000 7efdaaa00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 0x7efd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:40.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.673+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7efda40ac3c0 con 0x7efdb406ce80 2024-04-26T20:01:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.673+0000 7efda9400640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 0x7efd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.674+0000 7efda9400640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 0x7efd9006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdb415a9c0 tx=0x7efd9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:40.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.679+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efda404b030 con 0x7efdb406ce80 2024-04-26T20:01:40.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.920+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdb40af9a0 con 0x7efdb406ce80 2024-04-26T20:01:40.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.922+0000 7efdaaa00640 1 -- 172.21.15.88:0/497393553 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5181 (secure 0 0 0) 0x7efda407a670 con 0x7efdb406ce80 2024-04-26T20:01:40.923 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:40.923 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/809249379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":809249379},{"type":"v1","addr":"172.21.15.88:6837","nonce":809249379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:01:05.341746+0000","modified":"2024-04-26T20:01:08.420438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4708":{"gid":4708,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.88:6836/351605980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":351605980},{"type":"v1","addr":"172.21.15.88:6836","nonce":351605980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4708,"qdb_cluster":[4708]},"id":2}]} 2024-04-26T20:01:40.923 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-04-26T20:01:40.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.924+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 msgr2=0x7efd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.924+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 0x7efd9006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efdb415a9c0 tx=0x7efd9c009040 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.924+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 msgr2=0x7efdb4148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.924+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efda4000c00 tx=0x7efda40046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 shutdown_connections 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efd9006c8f0 0x7efd9006edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efdb415cdf0 0x7efdb415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efdb40cd650 0x7efdb415fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 --2- 172.21.15.88:0/497393553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efdb406ce80 0x7efdb4148640 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 >> 172.21.15.88:0/497393553 conn(0x7efdb40b2d50 msgr2=0x7efdb40b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:40.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 shutdown_connections 2024-04-26T20:01:40.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:40.925+0000 7efdb8e00640 1 -- 172.21.15.88:0/497393553 wait complete. 2024-04-26T20:01:40.944 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:41.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.143+0000 7f0299400640 1 Processor -- start 2024-04-26T20:01:41.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.143+0000 7f0299400640 1 -- start start 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0299400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0294146680 0x7f02940b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0299400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940b4b30 0x7f02940d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0299400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0294146a60 con 0x7f02940d0e70 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0299400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f029406e970 con 0x7f0294146680 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0299400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f02940d17f0 con 0x7f02940b4b30 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0293e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0294146680 0x7f02940b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0293e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0294146680 0x7f02940b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46184/0 (socket says 172.21.15.88:46184) 2024-04-26T20:01:41.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.145+0000 7f0293e00640 1 -- 172.21.15.88:0/541756186 learned_addr learned my addr 172.21.15.88:0/541756186 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.146+0000 7f0298a00640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940b4b30 0x7f02940d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.146+0000 7f0293e00640 1 -- 172.21.15.88:0/541756186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940b4b30 msgr2=0x7f02940d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.146+0000 7f0293e00640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940b4b30 0x7f02940d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.146+0000 7f0293e00640 1 -- 172.21.15.88:0/541756186 >> v1:172.21.15.132:6789/0 conn(0x7f02940d0e70 legacy=0x7f0294160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.146+0000 7f0293e00640 1 -- 172.21.15.88:0/541756186 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029406dd60 con 0x7f0294146680 2024-04-26T20:01:41.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.147+0000 7f0293e00640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f02940b5990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0284009870 tx=0x7f028402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4497e2398ce5d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.147+0000 7f0292a00640 1 -- 172.21.15.88:0/541756186 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284037070 con 0x7f0294146680 2024-04-26T20:01:41.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.147+0000 7f0292a00640 1 -- 172.21.15.88:0/541756186 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0284032440 con 0x7f0294146680 2024-04-26T20:01:41.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.147+0000 7f0292a00640 1 -- 172.21.15.88:0/541756186 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284032810 con 0x7f0294146680 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 -- 172.21.15.88:0/541756186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 msgr2=0x7f02940b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f02940b5990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0284009870 tx=0x7f028402a210 comp rx=0 tx=0).stop 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 -- 172.21.15.88:0/541756186 shutdown_connections 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940b4b30 0x7f02940d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 --2- 172.21.15.88:0/541756186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f02940b5990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 -- 172.21.15.88:0/541756186 >> 172.21.15.88:0/541756186 conn(0x7f029406c740 msgr2=0x7f029406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 -- 172.21.15.88:0/541756186 shutdown_connections 2024-04-26T20:01:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.148+0000 7f0299400640 1 -- 172.21.15.88:0/541756186 wait complete. 2024-04-26T20:01:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.150+0000 7f0299400640 1 Processor -- start 2024-04-26T20:01:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.150+0000 7f0299400640 1 -- start start 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02940b4b30 0x7f029413b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940d0e70 0x7f029413ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f029406e600 con 0x7f0294146680 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f029406e320 con 0x7f02940b4b30 2024-04-26T20:01:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.151+0000 7f0299400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f02940cd6a0 con 0x7f02940d0e70 2024-04-26T20:01:41.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0293e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02940b4b30 0x7f029413b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46186/0 (socket says 172.21.15.88:46186) 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 -- 172.21.15.88:0/2967280380 learned_addr learned my addr 172.21.15.88:0/2967280380 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0293400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940d0e70 0x7f029413ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 -- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940d0e70 msgr2=0x7f029413ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940d0e70 0x7f029413ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 -- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02940b4b30 msgr2=0x7f029413b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02940b4b30 0x7f029413b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.152+0000 7f0298a00640 1 -- 172.21.15.88:0/2967280380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029413a8e0 con 0x7f0294146680 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f0298a00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f028c007de0 tx=0x7f028c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c014620 con 0x7f0294146680 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c019070 con 0x7f0294146680 2024-04-26T20:01:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029413f6c0 con 0x7f0294146680 2024-04-26T20:01:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c014a20 con 0x7f0294146680 2024-04-26T20:01:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.153+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029413fc80 con 0x7f0294146680 2024-04-26T20:01:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.154+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029413fe80 con 0x7f0294146680 2024-04-26T20:01:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.155+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f028c014bc0 con 0x7f0294146680 2024-04-26T20:01:41.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.156+0000 7f028be00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.156+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7f028c08c410 con 0x7f0294146680 2024-04-26T20:01:41.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.156+0000 7f0293e00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.157+0000 7f0293e00640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 0x7f027006ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0284034000 tx=0x7f028402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.161+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f029413fe80 con 0x7f0294146680 2024-04-26T20:01:41.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.361+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02940693c0 con 0x7f0294146680 2024-04-26T20:01:41.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.364+0000 7f028be00640 1 -- 172.21.15.88:0/2967280380 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16786 (secure 0 0 0) 0x7f028c05a7d0 con 0x7f0294146680 2024-04-26T20:01:41.365 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:41.365 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:31.270527+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:03.091802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:01:04.331500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 msgr2=0x7f027006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 0x7f027006ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0284034000 tx=0x7f028402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 msgr2=0x7f029413d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f028c007de0 tx=0x7f028c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 shutdown_connections 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0294146680 0x7f029413d450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f027006c920 0x7f027006ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f02940d0e70 0x7f029413ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 --2- 172.21.15.88:0/2967280380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02940b4b30 0x7f029413b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 >> 172.21.15.88:0/2967280380 conn(0x7f029406c740 msgr2=0x7f02940b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:41.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 shutdown_connections 2024-04-26T20:01:41.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.367+0000 7f0299400640 1 -- 172.21.15.88:0/2967280380 wait complete. 2024-04-26T20:01:41.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:01:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.588+0000 7fd39b600640 1 Processor -- start 2024-04-26T20:01:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.588+0000 7fd39b600640 1 -- start start 2024-04-26T20:01:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd39b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 0x7fd3940add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd39b600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3940cd650 0x7fd3940b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd39b600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3940af9a0 con 0x7fd39415cdf0 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd39b600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd3940ae2c0 con 0x7fd3940cd650 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd39b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd3940b05b0 con 0x7fd39406ce80 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd399600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd3940cd650 0x7fd3940b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd38be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 0x7fd3940add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.590+0000 7fd398c00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fd39415cdf0 0x7fd39406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:44160/0 (socket says 172.21.15.88:44160) 2024-04-26T20:01:41.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd398c00640 1 -- 172.21.15.88:0/3029478037 learned_addr learned my addr 172.21.15.88:0/3029478037 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd393e00640 1 -- 172.21.15.88:0/3029478037 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341803739 0 0) 0x7fd3940af9a0 con 0x7fd39415cdf0 2024-04-26T20:01:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd393e00640 1 -- 172.21.15.88:0/3029478037 --> v1:172.21.15.132:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd378003520 con 0x7fd39415cdf0 2024-04-26T20:01:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd38be00640 1 -- 172.21.15.88:0/3029478037 >> v2:172.21.15.132:3300/0 conn(0x7fd3940cd650 msgr2=0x7fd3940b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd38be00640 1 --2- 172.21.15.88:0/3029478037 >> v2:172.21.15.132:3300/0 conn(0x7fd3940cd650 0x7fd3940b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.591+0000 7fd38be00640 1 -- 172.21.15.88:0/3029478037 >> v1:172.21.15.132:6789/0 conn(0x7fd39415cdf0 legacy=0x7fd39406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.592+0000 7fd38be00640 1 -- 172.21.15.88:0/3029478037 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3940afc80 con 0x7fd39406ce80 2024-04-26T20:01:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.593+0000 7fd38be00640 1 --2- 172.21.15.88:0/3029478037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 0x7fd3940add80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097a0 tx=0x7fd37c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973c91a6b60cf5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.593+0000 7fd393e00640 1 -- 172.21.15.88:0/3029478037 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c037070 con 0x7fd39406ce80 2024-04-26T20:01:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.593+0000 7fd393e00640 1 -- 172.21.15.88:0/3029478037 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c032440 con 0x7fd39406ce80 2024-04-26T20:01:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.593+0000 7fd393e00640 1 -- 172.21.15.88:0/3029478037 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0327f0 con 0x7fd39406ce80 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 -- 172.21.15.88:0/3029478037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 msgr2=0x7fd3940add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 --2- 172.21.15.88:0/3029478037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 0x7fd3940add80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097a0 tx=0x7fd37c02a180 comp rx=0 tx=0).stop 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 -- 172.21.15.88:0/3029478037 shutdown_connections 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 --2- 172.21.15.88:0/3029478037 >> v2:172.21.15.132:3300/0 conn(0x7fd3940cd650 0x7fd3940b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 --2- 172.21.15.88:0/3029478037 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd39406ce80 0x7fd3940add80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 -- 172.21.15.88:0/3029478037 >> 172.21.15.88:0/3029478037 conn(0x7fd3940b2d50 msgr2=0x7fd3940b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 -- 172.21.15.88:0/3029478037 shutdown_connections 2024-04-26T20:01:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.594+0000 7fd39b600640 1 -- 172.21.15.88:0/3029478037 wait complete. 2024-04-26T20:01:41.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.596+0000 7fd39b600640 1 Processor -- start 2024-04-26T20:01:41.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.596+0000 7fd39b600640 1 -- start start 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3940cd650 0x7fd39409c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 0x7fd39414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd39415d1d0 con 0x7fd39406ce80 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd3940b0240 con 0x7fd39415cdf0 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd39b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd39415d350 con 0x7fd3940cd650 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd398c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd398c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46192/0 (socket says 172.21.15.88:46192) 2024-04-26T20:01:41.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.597+0000 7fd398c00640 1 -- 172.21.15.88:0/1924321369 learned_addr learned my addr 172.21.15.88:0/1924321369 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd38be00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3940cd650 0x7fd39409c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd399600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 0x7fd39414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 -- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3940cd650 msgr2=0x7fd39409c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3940cd650 0x7fd39409c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 -- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 msgr2=0x7fd39414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 0x7fd39414b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 -- 172.21.15.88:0/1924321369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39414bc40 con 0x7fd39406ce80 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd399600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 0x7fd39414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd398c00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c390 tx=0x7fd38400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.598+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384017070 con 0x7fd39406ce80 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.599+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd384005ce0 con 0x7fd39406ce80 2024-04-26T20:01:41.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.599+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39409cbd0 con 0x7fd39406ce80 2024-04-26T20:01:41.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.599+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840072f0 con 0x7fd39406ce80 2024-04-26T20:01:41.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.599+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39409d190 con 0x7fd39406ce80 2024-04-26T20:01:41.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.600+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd394069540 con 0x7fd39406ce80 2024-04-26T20:01:41.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.601+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd38401a070 con 0x7fd39406ce80 2024-04-26T20:01:41.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.602+0000 7fd391600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 0x7fd37006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:41.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.602+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5795+0+0 (secure 0 0 0) 0x7fd38408afd0 con 0x7fd39406ce80 2024-04-26T20:01:41.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.603+0000 7fd38be00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 0x7fd37006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:41.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.603+0000 7fd38be00640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 0x7fd37006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd37c034000 tx=0x7fd37c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:41.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.607+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd39414bc40 con 0x7fd39406ce80 2024-04-26T20:01:41.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.852+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd3940ade20 con 0x7fd39406ce80 2024-04-26T20:01:41.854 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-04-26T20:01:41.853+0000 7fad57a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:01:41.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.857+0000 7fd391600640 1 -- 172.21.15.88:0/1924321369 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fd3840593a0 con 0x7fd39406ce80 2024-04-26T20:01:41.858 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:01:41.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 msgr2=0x7fd37006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 0x7fd37006edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd37c034000 tx=0x7fd37c02e000 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 msgr2=0x7fd3940b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd38400c390 tx=0x7fd38400c850 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 shutdown_connections 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd39415cdf0 0x7fd39414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd37006c8f0 0x7fd37006edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3940cd650 0x7fd39409c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 --2- 172.21.15.88:0/1924321369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd39406ce80 0x7fd3940b21c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 >> 172.21.15.88:0/1924321369 conn(0x7fd3940b2d50 msgr2=0x7fd39406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:41.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 shutdown_connections 2024-04-26T20:01:41.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:41.861+0000 7fd39b600640 1 -- 172.21.15.88:0/1924321369 wait complete. 2024-04-26T20:01:41.871 INFO:tasks.ceph.mds.a.smithi088.stderr: -1892> 2024-04-26T20:01:08.809+0000 7f4d03e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:01:41.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:01:41.925 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:01:42.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.058+0000 7ff3cec00640 1 Processor -- start 2024-04-26T20:01:42.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.058+0000 7ff3cec00640 1 -- start start 2024-04-26T20:01:42.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.059+0000 7ff3cec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff3c806ce80 0x7ff3c80b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.059+0000 7ff3cec00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c80af800 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.059+0000 7ff3cec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c80ae340 con 0x7ff3c815d100 2024-04-26T20:01:42.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.059+0000 7ff3cec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c80afaa0 con 0x7ff3c806ce80 2024-04-26T20:01:42.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3ccc00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff3c80b5ca0 0x7ff3c80b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41494/0 (socket says 172.21.15.88:41494) 2024-04-26T20:01:42.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3ccc00640 1 -- 172.21.15.88:0/2118981697 learned_addr learned my addr 172.21.15.88:0/2118981697 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2118981697 >> v2:172.21.15.132:3300/0 conn(0x7ff3c806ce80 0x7ff3c80b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996660760 0 0) 0x7ff3c80af800 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3ac003520 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.060+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3554801910 0 0) 0x7ff3ac003520 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 >> v2:172.21.15.132:3300/0 conn(0x7ff3c806ce80 msgr2=0x7ff3c80b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 --2- 172.21.15.88:0/2118981697 >> v2:172.21.15.132:3300/0 conn(0x7ff3c806ce80 0x7ff3c80b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 >> v1:172.21.15.132:6789/0 conn(0x7ff3c815d100 legacy=0x7ff3c806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c80afd80 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7ff3c00019f0 con 0x7ff3c80b5ca0 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 >> v1:172.21.15.88:6789/0 conn(0x7ff3c80b5ca0 legacy=0x7ff3c80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3ac0039b0 0x7ff3ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 0x7ff3ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac0051c0 0x7ff3ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c80afaa0 con 0x7ff3ac004800 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac00c160 con 0x7ff3ac0039b0 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ac00c360 con 0x7ff3ac0051c0 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 0x7ff3ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3c7e00640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3ac0039b0 0x7ff3ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3ccc00640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac0051c0 0x7ff3ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3bf400640 1 -- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac0051c0 msgr2=0x7ff3ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac0051c0 0x7ff3ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3bf400640 1 -- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3ac0039b0 msgr2=0x7ff3ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.061+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3ac0039b0 0x7ff3ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.062+0000 7ff3bf400640 1 -- 172.21.15.88:0/2118981697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac003520 con 0x7ff3ac004800 2024-04-26T20:01:42.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.062+0000 7ff3ccc00640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac0051c0 0x7ff3ac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.062+0000 7ff3ccc00640 1 -- 172.21.15.88:0/2118981697 reap_dead start 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 0x7ff3ac004c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009ac0 tx=0x7ff3ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2811769684f2969 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0036070 con 0x7ff3ac004800 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b0031400 con 0x7ff3ac004800 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3c7400640 1 -- 172.21.15.88:0/2118981697 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0009690 con 0x7ff3ac004800 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3cec00640 1 -- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 msgr2=0x7ff3ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 0x7ff3ac004c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009ac0 tx=0x7ff3ac005830 comp rx=0 tx=0).stop 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3cec00640 1 -- 172.21.15.88:0/2118981697 shutdown_connections 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2118981697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3ac004800 0x7ff3ac004c80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.063+0000 7ff3cec00640 1 -- 172.21.15.88:0/2118981697 >> 172.21.15.88:0/2118981697 conn(0x7ff3c80b2d50 msgr2=0x7ff3c80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.064+0000 7ff3cec00640 1 -- 172.21.15.88:0/2118981697 shutdown_connections 2024-04-26T20:01:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.064+0000 7ff3cec00640 1 -- 172.21.15.88:0/2118981697 wait complete. 2024-04-26T20:01:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.065+0000 7ff3cec00640 1 Processor -- start 2024-04-26T20:01:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.065+0000 7ff3cec00640 1 -- start start 2024-04-26T20:01:42.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac004800 0x7ff3c81486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3c815d9d0 0x7ff3c809d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c809ee50 con 0x7ff3c815d550 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c80b0340 con 0x7ff3c815d9d0 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.066+0000 7ff3cec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c8160b70 con 0x7ff3ac004800 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46224/0 (socket says 172.21.15.88:46224) 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 -- 172.21.15.88:0/2810172971 learned_addr learned my addr 172.21.15.88:0/2810172971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3c7e00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac004800 0x7ff3c81486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 -- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac004800 msgr2=0x7ff3c81486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3ccc00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3c815d9d0 0x7ff3c809d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac004800 0x7ff3c81486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 -- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3c815d9d0 msgr2=0x7ff3c809d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3c815d9d0 0x7ff3c809d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 -- 172.21.15.88:0/2810172971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c815b140 con 0x7ff3c815d550 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.067+0000 7ff3bf400640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0034040 tx=0x7ff3b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.068+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b003f020 con 0x7ff3c815d550 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.068+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b0037040 con 0x7ff3c815d550 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.068+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0036070 con 0x7ff3c815d550 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.068+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c815b420 con 0x7ff3c815d550 2024-04-26T20:01:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.068+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c815b9e0 con 0x7ff3c815d550 2024-04-26T20:01:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.069+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c8069650 con 0x7ff3c815d550 2024-04-26T20:01:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.070+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3b0042020 con 0x7ff3c815d550 2024-04-26T20:01:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.071+0000 7ff3c4c00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.071+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5881+0+0 (secure 0 0 0) 0x7ff3b00abd60 con 0x7ff3c815d550 2024-04-26T20:01:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.071+0000 7ff3c7e00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.072+0000 7ff3c7e00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 0x7ff3a406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:42.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.077+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3b0049030 con 0x7ff3c815d550 2024-04-26T20:01:42.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.317+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff3c8073d00 con 0x7ff3c815d550 2024-04-26T20:01:42.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.861+0000 7ff3c4c00640 1 -- 172.21.15.88:0/2810172971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7ff3b007a100 con 0x7ff3c815d550 2024-04-26T20:01:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.864+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 msgr2=0x7ff3a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.864+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 0x7ff3a406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8009040 comp rx=0 tx=0).stop 2024-04-26T20:01:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.864+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 msgr2=0x7ff3c809d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.864+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0034040 tx=0x7ff3b0004730 comp rx=0 tx=0).stop 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 shutdown_connections 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3c815d9d0 0x7ff3c809d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3c815d550 0x7ff3c809d250 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff3a406c8f0 0x7ff3a406edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 --2- 172.21.15.88:0/2810172971 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3ac004800 0x7ff3c81486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 >> 172.21.15.88:0/2810172971 conn(0x7ff3c80b2d50 msgr2=0x7ff3c80b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 shutdown_connections 2024-04-26T20:01:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:42.865+0000 7ff3cec00640 1 -- 172.21.15.88:0/2810172971 wait complete. 2024-04-26T20:01:42.889 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:01:43.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.143+0000 7f7ddf200640 1 Processor -- start 2024-04-26T20:01:43.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.143+0000 7f7ddf200640 1 -- start start 2024-04-26T20:01:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.144+0000 7f7ddf200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7de00d3220 0x7f7de0169c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.144+0000 7f7ddf200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de015f530 con 0x7f7de00d55c0 2024-04-26T20:01:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.144+0000 7f7ddf200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de00c8a90 con 0x7f7de015f150 2024-04-26T20:01:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.144+0000 7f7ddf200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f7de00d37d0 con 0x7f7de00d3220 2024-04-26T20:01:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dd5400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7de00d55c0 0x7f7de00d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41508/0 (socket says 172.21.15.88:41508) 2024-04-26T20:01:43.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dd5400640 1 -- 172.21.15.88:0/2359687999 learned_addr learned my addr 172.21.15.88:0/2359687999 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:43.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dde800640 1 --2- 172.21.15.88:0/2359687999 >> v2:172.21.15.88:3300/0 conn(0x7f7de00d3220 0x7f7de0169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:43.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7ddd400640 1 -- 172.21.15.88:0/2359687999 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551277834 0 0) 0x7f7dc8001110 con 0x7f7de00d55c0 2024-04-26T20:01:43.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7ddd400640 1 -- 172.21.15.88:0/2359687999 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dc4003520 con 0x7f7de00d55c0 2024-04-26T20:01:43.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dde800640 1 -- 172.21.15.88:0/2359687999 >> v1:172.21.15.132:6789/0 conn(0x7f7de015f150 legacy=0x7f7de00d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:43.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dde800640 1 -- 172.21.15.88:0/2359687999 >> v1:172.21.15.88:6789/0 conn(0x7f7de00d55c0 legacy=0x7f7de00d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:43.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.145+0000 7f7dde800640 1 -- 172.21.15.88:0/2359687999 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de00d3ab0 con 0x7f7de00d3220 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.146+0000 7f7dde800640 1 --2- 172.21.15.88:0/2359687999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d3220 0x7f7de0169c50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097a0 tx=0x7f7dd802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad39b1aa98a1463 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddd400640 1 -- 172.21.15.88:0/2359687999 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8037070 con 0x7f7de00d3220 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddd400640 1 -- 172.21.15.88:0/2359687999 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd802ae50 con 0x7f7de00d3220 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddd400640 1 -- 172.21.15.88:0/2359687999 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8032660 con 0x7f7de00d3220 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 -- 172.21.15.88:0/2359687999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d3220 msgr2=0x7f7de0169c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 --2- 172.21.15.88:0/2359687999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d3220 0x7f7de0169c50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097a0 tx=0x7f7dd802a140 comp rx=0 tx=0).stop 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 -- 172.21.15.88:0/2359687999 shutdown_connections 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 --2- 172.21.15.88:0/2359687999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d3220 0x7f7de0169c50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 -- 172.21.15.88:0/2359687999 >> 172.21.15.88:0/2359687999 conn(0x7f7de008e990 msgr2=0x7f7de009e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:43.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 -- 172.21.15.88:0/2359687999 shutdown_connections 2024-04-26T20:01:43.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.147+0000 7f7ddf200640 1 -- 172.21.15.88:0/2359687999 wait complete. 2024-04-26T20:01:43.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.149+0000 7f7ddf200640 1 Processor -- start 2024-04-26T20:01:43.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.149+0000 7f7ddf200640 1 -- start start 2024-04-26T20:01:43.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7de00d3220 0x7f7de00ba400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:43.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7de015f150 0x7f7de00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00d1340 con 0x7f7de00d55c0 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00cfac0 con 0x7f7de015f150 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7ddf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7de016a190 con 0x7f7de00d3220 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7dd5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7dd5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46252/0 (socket says 172.21.15.88:46252) 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.150+0000 7f7dd5400640 1 -- 172.21.15.88:0/1019863595 learned_addr learned my addr 172.21.15.88:0/1019863595 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 -- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7de00d3220 msgr2=0x7f7de00ba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7de00d3220 0x7f7de00ba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 -- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7de015f150 msgr2=0x7f7de00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7de015f150 0x7f7de00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 -- 172.21.15.88:0/1019863595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de015a5e0 con 0x7f7de00d55c0 2024-04-26T20:01:43.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd5400640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c730 tx=0x7f7dc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8017070 con 0x7f7de00d55c0 2024-04-26T20:01:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc8005ad0 con 0x7f7de00d55c0 2024-04-26T20:01:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de00b27e0 con 0x7f7de00d55c0 2024-04-26T20:01:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.151+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007e20 con 0x7f7de00d55c0 2024-04-26T20:01:43.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.152+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de00b85d0 con 0x7f7de00d55c0 2024-04-26T20:01:43.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.153+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de00693c0 con 0x7f7de00d55c0 2024-04-26T20:01:43.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.154+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dc801a070 con 0x7f7de00d55c0 2024-04-26T20:01:43.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.155+0000 7f7dd6a00640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 0x7f7dbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:43.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.155+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5881+0+0 (secure 0 0 0) 0x7f7dc808b060 con 0x7f7de00d55c0 2024-04-26T20:01:43.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.155+0000 7f7ddde00640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 0x7f7dbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:43.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.156+0000 7f7ddde00640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 0x7f7dbc06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7de008d2c0 tx=0x7f7dd0007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.160+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7de015a5e0 con 0x7f7de00d55c0 2024-04-26T20:01:43.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:43.360+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7de00d1100 con 0x7f7de00d55c0 2024-04-26T20:01:44.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.371+0000 7f7dd6a00640 1 -- 172.21.15.88:0/1019863595 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7f7de015f530 con 0x7f7de00d55c0 2024-04-26T20:01:44.372 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:01:44.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.375+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 msgr2=0x7f7dbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:44.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.375+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 0x7f7dbc06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7de008d2c0 tx=0x7f7dd0007d30 comp rx=0 tx=0).stop 2024-04-26T20:01:44.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.375+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 msgr2=0x7f7de0159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:44.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.375+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c730 tx=0x7f7dc800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 shutdown_connections 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7de015f150 0x7f7de00ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7de00d55c0 0x7f7de0159c20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7dbc06c8f0 0x7f7dbc06edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 --2- 172.21.15.88:0/1019863595 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7de00d3220 0x7f7de00ba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.376+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 >> 172.21.15.88:0/1019863595 conn(0x7f7de008e990 msgr2=0x7f7de00d4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:44.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.377+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 shutdown_connections 2024-04-26T20:01:44.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.377+0000 7f7ddf200640 1 -- 172.21.15.88:0/1019863595 wait complete. 2024-04-26T20:01:44.408 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:01:44.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.596+0000 7f870ee00640 1 Processor -- start 2024-04-26T20:01:44.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.596+0000 7f870ee00640 1 -- start start 2024-04-26T20:01:44.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.597+0000 7f870ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870815d170 0x7f87080cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.597+0000 7f870ee00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87080aff10 0x7f8708072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.597+0000 7f870ee00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87080ae160 con 0x7f870806ef60 2024-04-26T20:01:44.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ee00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f87080d0520 con 0x7f87080aff10 2024-04-26T20:01:44.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f870815d550 con 0x7f870815d170 2024-04-26T20:01:44.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87080aff10 0x7f8708072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87080aff10 0x7f8708072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46268/0 (socket says 172.21.15.88:46268) 2024-04-26T20:01:44.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 -- 172.21.15.88:0/734460000 learned_addr learned my addr 172.21.15.88:0/734460000 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f8707e00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870815d170 0x7f87080cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 -- 172.21.15.88:0/734460000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870815d170 msgr2=0x7f87080cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870815d170 0x7f87080cffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 -- 172.21.15.88:0/734460000 >> v1:172.21.15.132:6789/0 conn(0x7f870806ef60 legacy=0x7f87080af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.598+0000 7f870ce00640 1 -- 172.21.15.88:0/734460000 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870806ff80 con 0x7f87080aff10 2024-04-26T20:01:44.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.599+0000 7f870ce00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f8708072030 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e50 tx=0x7f870002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a94cbfcd393b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f8707400640 1 -- 172.21.15.88:0/734460000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700036070 con 0x7f87080aff10 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f8707400640 1 -- 172.21.15.88:0/734460000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8700031600 con 0x7f87080aff10 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f8707400640 1 -- 172.21.15.88:0/734460000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87000319d0 con 0x7f87080aff10 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 -- 172.21.15.88:0/734460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 msgr2=0x7f8708072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f8708072030 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e50 tx=0x7f870002a640 comp rx=0 tx=0).stop 2024-04-26T20:01:44.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 -- 172.21.15.88:0/734460000 shutdown_connections 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f8708072030 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 --2- 172.21.15.88:0/734460000 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870815d170 0x7f87080cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 -- 172.21.15.88:0/734460000 >> 172.21.15.88:0/734460000 conn(0x7f87080d0c40 msgr2=0x7f87080d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 -- 172.21.15.88:0/734460000 shutdown_connections 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.600+0000 7f870ee00640 1 -- 172.21.15.88:0/734460000 wait complete. 2024-04-26T20:01:44.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.602+0000 7f870ee00640 1 Processor -- start 2024-04-26T20:01:44.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.602+0000 7f870ee00640 1 -- start start 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 0x7f870815ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 0x7f87080b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f870806ea40 con 0x7f87080aff10 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f870806fd30 con 0x7f870815d170 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f870ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f87080add80 con 0x7f870806ef60 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f86ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.603+0000 7f86ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46282/0 (socket says 172.21.15.88:46282) 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 -- 172.21.15.88:0/2922488162 learned_addr learned my addr 172.21.15.88:0/2922488162 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f8707e00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 0x7f870815ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f870ce00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 0x7f87080b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 -- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 msgr2=0x7f870815ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:44.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 0x7f870815ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 -- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 msgr2=0x7f87080b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 0x7f87080b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 -- 172.21.15.88:0/2922488162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8708153d80 con 0x7f87080aff10 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f870ce00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 0x7f87080b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f8707e00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 0x7f870815ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.604+0000 7f86ff400640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c460 tx=0x7f86f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.605+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0016020 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.605+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f0012870 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.605+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0012c60 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.605+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870815d6d0 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.605+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870815dc90 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.606+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8708069650 con 0x7f87080aff10 2024-04-26T20:01:44.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.607+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86f0019020 con 0x7f87080aff10 2024-04-26T20:01:44.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.608+0000 7f8704c00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 0x7f86e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:44.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.608+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5443+0+0 (secure 0 0 0) 0x7f86f008b850 con 0x7f87080aff10 2024-04-26T20:01:44.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.608+0000 7f8707e00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 0x7f86e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:44.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.609+0000 7f8707e00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 0x7f86e406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b820 tx=0x7f86f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:44.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.612+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86f0020030 con 0x7f87080aff10 2024-04-26T20:01:44.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:44.814+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f870809ee70 con 0x7f87080aff10 2024-04-26T20:01:45.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.382+0000 7f8704c00640 1 -- 172.21.15.88:0/2922488162 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7f86f0019310 con 0x7f87080aff10 2024-04-26T20:01:45.383 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-04-26T20:01:45.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.386+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 msgr2=0x7f86e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.386+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 0x7f86e406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b820 tx=0x7f86f8008040 comp rx=0 tx=0).stop 2024-04-26T20:01:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.386+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 msgr2=0x7f87081495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.386+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c460 tx=0x7f86f000c920 comp rx=0 tx=0).stop 2024-04-26T20:01:45.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 shutdown_connections 2024-04-26T20:01:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f870815d170 0x7f87080b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87080aff10 0x7f87081495e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f86e406c8f0 0x7f86e406edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 --2- 172.21.15.88:0/2922488162 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f870806ef60 0x7f870815ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.387+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 >> 172.21.15.88:0/2922488162 conn(0x7f87080d0c40 msgr2=0x7f87080cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:45.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.388+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 shutdown_connections 2024-04-26T20:01:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.390+0000 7f870ee00640 1 -- 172.21.15.88:0/2922488162 wait complete. 2024-04-26T20:01:45.419 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:01:45.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.600+0000 7f1a0d600640 1 Processor -- start 2024-04-26T20:01:45.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.600+0000 7f1a0d600640 1 -- start start 2024-04-26T20:01:45.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f1a080d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0d600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1a0815de80 0x7f1a0815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0d600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a08156750 con 0x7f1a08156370 2024-04-26T20:01:45.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0d600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0815d430 con 0x7f1a0815de80 2024-04-26T20:01:45.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0815cd60 con 0x7f1a080d0d60 2024-04-26T20:01:45.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a07e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1a08156370 0x7f1a0815c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41522/0 (socket says 172.21.15.88:41522) 2024-04-26T20:01:45.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a07e00640 1 -- 172.21.15.88:0/3644780420 learned_addr learned my addr 172.21.15.88:0/3644780420 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f19ff400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f1a080d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.602+0000 7f1a0cc00640 1 --2- 172.21.15.88:0/3644780420 >> v2:172.21.15.132:3300/0 conn(0x7f1a0815de80 0x7f1a0815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406625961 0 0) 0x7f19f8001110 con 0x7f1a08156370 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19ec003520 con 0x7f1a08156370 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3535930 0 0) 0x7f19ec003520 con 0x7f1a08156370 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 msgr2=0x7f1a080d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f1a080d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 >> v2:172.21.15.132:3300/0 conn(0x7f1a0815de80 msgr2=0x7f1a0815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 --2- 172.21.15.88:0/3644780420 >> v2:172.21.15.132:3300/0 conn(0x7f1a0815de80 0x7f1a0815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0815d040 con 0x7f1a08156370 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f19f8001840 con 0x7f1a08156370 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 >> v1:172.21.15.88:6789/0 conn(0x7f1a08156370 legacy=0x7f1a0815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f19ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0815d430 con 0x7f19ec0039b0 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0815cd60 con 0x7f19ec004800 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec00dbd0 con 0x7f1a080d0d60 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.603+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f19ff400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a07e00640 1 -- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 msgr2=0x7f19ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a0cc00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f19ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f19ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a07e00640 1 -- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec004800 msgr2=0x7f19ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:45.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a07e00640 1 -- 172.21.15.88:0/3644780420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec003520 con 0x7f19ec0039b0 2024-04-26T20:01:45.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f19ff400640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec004800 0x7f19ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:45.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.604+0000 7f1a0cc00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a080d0d60 0x7f19ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:45.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.605+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c320 tx=0x7f19ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e34c2463c3025 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:45.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.605+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8005ad0 con 0x7f19ec0039b0 2024-04-26T20:01:45.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.605+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f8034070 con 0x7f19ec0039b0 2024-04-26T20:01:45.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.605+0000 7f1a07400640 1 -- 172.21.15.88:0/3644780420 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8007070 con 0x7f19ec0039b0 2024-04-26T20:01:45.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 -- 172.21.15.88:0/3644780420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19ec0039b0 msgr2=0x7f19ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3644780420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19ec0039b0 0x7f19ec003dd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c320 tx=0x7f19ec005830 comp rx=0 tx=0).stop 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0cc00640 1 -- 172.21.15.88:0/3644780420 reap_dead start 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 -- 172.21.15.88:0/3644780420 shutdown_connections 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 -- 172.21.15.88:0/3644780420 >> 172.21.15.88:0/3644780420 conn(0x7f1a0806c530 msgr2=0x7f1a0806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 -- 172.21.15.88:0/3644780420 shutdown_connections 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.606+0000 7f1a0d600640 1 -- 172.21.15.88:0/3644780420 wait complete. 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.608+0000 7f1a0d600640 1 Processor -- start 2024-04-26T20:01:45.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.608+0000 7f1a0d600640 1 -- start start 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec0039b0 0x7f1a081486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a081569a0 0x7f1a08156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a080b3cb0 con 0x7f1a081e6840 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0815d960 con 0x7f19ec0039b0 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a0d600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0815c930 con 0x7f1a081569a0 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f19ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f19ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46302/0 (socket says 172.21.15.88:46302) 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f19ff400640 1 -- 172.21.15.88:0/3453143367 learned_addr learned my addr 172.21.15.88:0/3453143367 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f19ff400640 1 -- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a081569a0 msgr2=0x7f1a08156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f19ff400640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a081569a0 0x7f1a08156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.609+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec0039b0 0x7f1a081486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f19ff400640 1 -- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec0039b0 msgr2=0x7f1a081486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f19ff400640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec0039b0 0x7f1a081486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f19ff400640 1 -- 172.21.15.88:0/3453143367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081576d0 con 0x7f1a081e6840 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f19ff400640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010830 tx=0x7f19f0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f000f040 con 0x7f1a081e6840 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f000eb00 con 0x7f1a081e6840 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a081585b0 con 0x7f1a081e6840 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f001c400 con 0x7f1a081e6840 2024-04-26T20:01:45.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.610+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a08158b70 con 0x7f1a081e6840 2024-04-26T20:01:45.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.611+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a081e7200 con 0x7f1a081e6840 2024-04-26T20:01:45.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.612+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19f001c5a0 con 0x7f1a081e6840 2024-04-26T20:01:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.614+0000 7f1a04c00640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.614+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7f19f008bf40 con 0x7f1a081e6840 2024-04-26T20:01:45.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.614+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.615+0000 7f1a07e00640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 0x7f19e406ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19f8033000 tx=0x7f19f8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:45.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.618+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19f0018030 con 0x7f1a081e6840 2024-04-26T20:01:45.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.821+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1a080693c0 con 0x7f1a081e6840 2024-04-26T20:01:45.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.827+0000 7f1a04c00640 1 -- 172.21.15.88:0/3453143367 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f19f005a570 con 0x7f1a081e6840 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 msgr2=0x7f19e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 0x7f19e406ede0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f19f8033000 tx=0x7f19f8009030 comp rx=0 tx=0).stop 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 msgr2=0x7f1a081e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010830 tx=0x7f19f0010cf0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 shutdown_connections 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f19e406c920 0x7f19e406ede0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a081569a0 0x7f1a08156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a081e6840 0x7f1a081e6cc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 --2- 172.21.15.88:0/3453143367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19ec0039b0 0x7f1a081486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:45.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 >> 172.21.15.88:0/3453143367 conn(0x7f1a0806c530 msgr2=0x7f1a08073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:45.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 shutdown_connections 2024-04-26T20:01:45.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:45.830+0000 7f1a0d600640 1 -- 172.21.15.88:0/3453143367 wait complete. 2024-04-26T20:01:45.850 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-04-26T20:01:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.041+0000 7f75c7800640 1 Processor -- start 2024-04-26T20:01:46.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.041+0000 7f75c7800640 1 -- start start 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c7800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75c006ce80 0x7f75c01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c7800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c015fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c7800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c00b5fb0 con 0x7f75c01598a0 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c7800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f75c0159c80 con 0x7f75c006ce80 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c7800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f75c01600f0 con 0x7f75c015f740 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75bfe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f75c006ce80 0x7f75c01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c4e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f75c01598a0 0x7f75c006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41538/0 (socket says 172.21.15.88:41538) 2024-04-26T20:01:46.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.043+0000 7f75c4e00640 1 -- 172.21.15.88:0/2559098079 learned_addr learned my addr 172.21.15.88:0/2559098079 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75c5800640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c015fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75bfe00640 1 -- 172.21.15.88:0/2559098079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 msgr2=0x7f75c015fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75bfe00640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c015fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75bfe00640 1 -- 172.21.15.88:0/2559098079 >> v1:172.21.15.88:6789/0 conn(0x7f75c01598a0 legacy=0x7f75c006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75bfe00640 1 -- 172.21.15.88:0/2559098079 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c0160370 con 0x7f75c006ce80 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75c5800640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c015fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.044+0000 7f75bf400640 1 -- 172.21.15.88:0/2559098079 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826435860 0 0) 0x7f75c00b5fb0 con 0x7f75c01598a0 2024-04-26T20:01:46.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.045+0000 7f75bfe00640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c01638b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0097a0 tx=0x7f75ac02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ba204170518fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.045+0000 7f75bf400640 1 -- 172.21.15.88:0/2559098079 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac037070 con 0x7f75c006ce80 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.045+0000 7f75bf400640 1 -- 172.21.15.88:0/2559098079 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ac02ae50 con 0x7f75c006ce80 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.045+0000 7f75bf400640 1 -- 172.21.15.88:0/2559098079 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac032660 con 0x7f75c006ce80 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 -- 172.21.15.88:0/2559098079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 msgr2=0x7f75c01638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c01638b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0097a0 tx=0x7f75ac02a100 comp rx=0 tx=0).stop 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 -- 172.21.15.88:0/2559098079 shutdown_connections 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c015fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 --2- 172.21.15.88:0/2559098079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c01638b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 -- 172.21.15.88:0/2559098079 >> 172.21.15.88:0/2559098079 conn(0x7f75c00b2d50 msgr2=0x7f75c00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 -- 172.21.15.88:0/2559098079 shutdown_connections 2024-04-26T20:01:46.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.046+0000 7f75c7800640 1 -- 172.21.15.88:0/2559098079 wait complete. 2024-04-26T20:01:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.047+0000 7f75c7800640 1 Processor -- start 2024-04-26T20:01:46.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.048+0000 7f75c7800640 1 -- start start 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.048+0000 7f75c7800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.048+0000 7f75c7800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c01598a0 0x7f75c01f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.048+0000 7f75c7800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.048+0000 7f75c7800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0163390 con 0x7f75c006ce80 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c7800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01630b0 con 0x7f75c01598a0 2024-04-26T20:01:46.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c7800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f75c0160650 con 0x7f75c015f740 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46322/0 (socket says 172.21.15.88:46322) 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 -- 172.21.15.88:0/2576608974 learned_addr learned my addr 172.21.15.88:0/2576608974 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 -- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 msgr2=0x7f75c00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75bfe00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c01598a0 0x7f75c01f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 -- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c01598a0 msgr2=0x7f75c01f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c01598a0 0x7f75c01f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.049+0000 7f75c4e00640 1 -- 172.21.15.88:0/2576608974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c01f8cf0 con 0x7f75c006ce80 2024-04-26T20:01:46.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75c4e00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c4d0 tx=0x7f75b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4018070 con 0x7f75c006ce80 2024-04-26T20:01:46.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b4007500 con 0x7f75c006ce80 2024-04-26T20:01:46.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4004a20 con 0x7f75c006ce80 2024-04-26T20:01:46.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c01f8fd0 con 0x7f75c006ce80 2024-04-26T20:01:46.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.050+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c00cb4f0 con 0x7f75c006ce80 2024-04-26T20:01:46.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.052+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c0069540 con 0x7f75c006ce80 2024-04-26T20:01:46.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.053+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75b401b070 con 0x7f75c006ce80 2024-04-26T20:01:46.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.055+0000 7f75bcc00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 0x7f75a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.055+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7f75b408ac90 con 0x7f75c006ce80 2024-04-26T20:01:46.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.055+0000 7f75bfe00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 0x7f75a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.056+0000 7f75bfe00640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 0x7f75a006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f75ac034000 tx=0x7f75ac02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.061+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75b40592c0 con 0x7f75c006ce80 2024-04-26T20:01:46.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.340+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) -- 0x7f75c01f91a0 con 0x7f75c006ce80 2024-04-26T20:01:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.385+0000 7f75bcc00640 1 -- 172.21.15.88:0/2576608974 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v135) ==== 135+0+0 (secure 0 0 0) 0x7f75b4058bc0 con 0x7f75c006ce80 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 msgr2=0x7f75a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 0x7f75a006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f75ac034000 tx=0x7f75ac02e000 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 msgr2=0x7f75c013f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f75b400c4d0 tx=0x7f75b400c990 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 shutdown_connections 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f75a006c8f0 0x7f75a006edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f75c015f740 0x7f75c00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c01598a0 0x7f75c01f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 --2- 172.21.15.88:0/2576608974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75c006ce80 0x7f75c013f960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 >> 172.21.15.88:0/2576608974 conn(0x7f75c00b2d50 msgr2=0x7f75c006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 shutdown_connections 2024-04-26T20:01:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.388+0000 7f75c7800640 1 -- 172.21.15.88:0/2576608974 wait complete. 2024-04-26T20:01:46.408 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2024-04-26T20:01:46.409 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2024-04-26T20:01:46.409 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:01:46.410 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:46.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.606+0000 7f8a70000640 1 Processor -- start 2024-04-26T20:01:46.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.606+0000 7f8a70000640 1 -- start start 2024-04-26T20:01:46.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a70000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a680d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a70000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8a6806d460 0x7f8a6815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a70000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a68072180 con 0x7f8a68071da0 2024-04-26T20:01:46.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a70000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8a6806d940 con 0x7f8a6806d460 2024-04-26T20:01:46.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a70000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6806dc20 con 0x7f8a6809c160 2024-04-26T20:01:46.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a6d600640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8a68071da0 0x7f8a6809ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:41542/0 (socket says 172.21.15.88:41542) 2024-04-26T20:01:46.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.608+0000 7f8a6d600640 1 -- 172.21.15.88:0/1073088649 learned_addr learned my addr 172.21.15.88:0/1073088649 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3628802128 0 0) 0x7f8a68072180 con 0x7f8a68071da0 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a50003520 con 0x7f8a68071da0 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a6e000640 1 --2- 172.21.15.88:0/1073088649 >> v2:172.21.15.132:3300/0 conn(0x7f8a6806d460 0x7f8a6815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a680d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1558180875 0 0) 0x7f8a50003520 con 0x7f8a68071da0 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 msgr2=0x7f8a680d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:46.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a680d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 >> v2:172.21.15.132:3300/0 conn(0x7f8a6806d460 msgr2=0x7f8a6815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 --2- 172.21.15.88:0/1073088649 >> v2:172.21.15.132:3300/0 conn(0x7f8a6806d460 0x7f8a6815eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6806df00 con 0x7f8a68071da0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f8a5c001670 con 0x7f8a68071da0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 >> v1:172.21.15.88:6789/0 conn(0x7f8a68071da0 legacy=0x7f8a6809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a500039b0 0x7f8a50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6806d940 con 0x7f8a50004800 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6806dc20 con 0x7f8a500039b0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5000ba70 con 0x7f8a6809c160 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.609+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6d600640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a500039b0 0x7f8a50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6e000640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6cc00640 1 -- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 msgr2=0x7f8a500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:46.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a6809c160 0x7f8a500051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6cc00640 1 -- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a500039b0 msgr2=0x7f8a50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:46.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a500039b0 0x7f8a50003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.610+0000 7f8a6cc00640 1 -- 172.21.15.88:0/1073088649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a50003520 con 0x7f8a50004800 2024-04-26T20:01:46.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.611+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a50004c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400ba00 tx=0x7f8a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c75fc94ba5bb154 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.611+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54030070 con 0x7f8a50004800 2024-04-26T20:01:46.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.611+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5402ad80 con 0x7f8a50004800 2024-04-26T20:01:46.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.611+0000 7f8a67e00640 1 -- 172.21.15.88:0/1073088649 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400b600 con 0x7f8a50004800 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 -- 172.21.15.88:0/1073088649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 msgr2=0x7f8a50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 --2- 172.21.15.88:0/1073088649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a50004c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400ba00 tx=0x7f8a50005830 comp rx=0 tx=0).stop 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a6e000640 1 -- 172.21.15.88:0/1073088649 reap_dead start 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 -- 172.21.15.88:0/1073088649 shutdown_connections 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 -- 172.21.15.88:0/1073088649 >> 172.21.15.88:0/1073088649 conn(0x7f8a680d2e70 msgr2=0x7f8a680d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 -- 172.21.15.88:0/1073088649 shutdown_connections 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.612+0000 7f8a70000640 1 -- 172.21.15.88:0/1073088649 wait complete. 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.614+0000 7f8a70000640 1 Processor -- start 2024-04-26T20:01:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.614+0000 7f8a70000640 1 -- start start 2024-04-26T20:01:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a680b6f20 0x7f8a680b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a680cd4a0 0x7f8a680cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a680d13f0 con 0x7f8a50004800 2024-04-26T20:01:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6806e4c0 con 0x7f8a680cd4a0 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a70000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a680d2040 con 0x7f8a680b6f20 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a6d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a6d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46336/0 (socket says 172.21.15.88:46336) 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.615+0000 7f8a6d600640 1 -- 172.21.15.88:0/1799260566 learned_addr learned my addr 172.21.15.88:0/1799260566 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a680b6f20 0x7f8a680b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 -- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a680b6f20 msgr2=0x7f8a680b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a680b6f20 0x7f8a680b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 -- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a680cd4a0 msgr2=0x7f8a680cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a680cd4a0 0x7f8a680cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 -- 172.21.15.88:0/1799260566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a680cf270 con 0x7f8a50004800 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a6d600640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d7f0 tx=0x7f8a5c00dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c019070 con 0x7f8a50004800 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c0074e0 con 0x7f8a50004800 2024-04-26T20:01:46.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c004a20 con 0x7f8a50004800 2024-04-26T20:01:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a680ce1d0 con 0x7f8a50004800 2024-04-26T20:01:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.616+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a680ce790 con 0x7f8a50004800 2024-04-26T20:01:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.617+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a68069540 con 0x7f8a50004800 2024-04-26T20:01:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.619+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a5c004bc0 con 0x7f8a50004800 2024-04-26T20:01:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.620+0000 7f8a65600640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.620+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7f8a5c08b300 con 0x7f8a50004800 2024-04-26T20:01:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.620+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:46.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.621+0000 7f8a6cc00640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 0x7f8a4806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a500057e0 tx=0x7f8a54030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:46.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.624+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a5c0592e0 con 0x7f8a50004800 2024-04-26T20:01:46.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.826+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a680d2420 con 0x7f8a50004800 2024-04-26T20:01:46.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.827+0000 7f8a65600640 1 -- 172.21.15.88:0/1799260566 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13136 (secure 0 0 0) 0x7f8a5c05d5e0 con 0x7f8a50004800 2024-04-26T20:01:46.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:46.828 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":43,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:45.372882+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6836/351605980":"2024-04-27T20:01:41.853988+0000","172.21.15.88:6834/351605980":"2024-04-27T20:01:41.853988+0000","172.21.15.88:6837/2458488289":"2024-04-27T20:01:28.259287+0000","172.21.15.88:6835/2458488289":"2024-04-27T20:01:28.259287+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 msgr2=0x7f8a4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 0x7f8a4806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a500057e0 tx=0x7f8a54030000 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 msgr2=0x7f8a681deb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d7f0 tx=0x7f8a5c00dcb0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 shutdown_connections 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a680cd4a0 0x7f8a680cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8a4806c8f0 0x7f8a4806edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a680b6f20 0x7f8a680b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 --2- 172.21.15.88:0/1799260566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a50004800 0x7f8a681deb20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 >> 172.21.15.88:0/1799260566 conn(0x7f8a680d2e70 msgr2=0x7f8a680b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 shutdown_connections 2024-04-26T20:01:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:46.830+0000 7f8a70000640 1 -- 172.21.15.88:0/1799260566 wait complete. 2024-04-26T20:01:46.849 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:46.850 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.030+0000 7fd8ea200640 1 Processor -- start 2024-04-26T20:01:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.030+0000 7fd8ea200640 1 -- start start 2024-04-26T20:01:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.031+0000 7fd8ea200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.031+0000 7fd8ea200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8e40d3c90 0x7fd8e40d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.031+0000 7fd8ea200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e40d0ed0 con 0x7fd8e40d1c20 2024-04-26T20:01:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.031+0000 7fd8ea200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e40d2540 con 0x7fd8e40d3c90 2024-04-26T20:01:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.031+0000 7fd8ea200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e40cfe30 con 0x7fd8e415f270 2024-04-26T20:01:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.032+0000 7fd8dbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd8e40d3c90 0x7fd8e40d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.032+0000 7fd8e8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.032+0000 7fd8e8e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:50066/0 (socket says 172.21.15.88:50066) 2024-04-26T20:01:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.032+0000 7fd8e8e00640 1 -- 172.21.15.88:0/2245230486 learned_addr learned my addr 172.21.15.88:0/2245230486 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.033+0000 7fd8e8e00640 1 -- 172.21.15.88:0/2245230486 >> v2:172.21.15.132:3300/0 conn(0x7fd8e40d3c90 msgr2=0x7fd8e40d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.033+0000 7fd8e8e00640 1 --2- 172.21.15.88:0/2245230486 >> v2:172.21.15.132:3300/0 conn(0x7fd8e40d3c90 0x7fd8e40d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.033+0000 7fd8e8e00640 1 -- 172.21.15.88:0/2245230486 >> v1:172.21.15.132:6789/0 conn(0x7fd8e40d1c20 legacy=0x7fd8e40b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.033+0000 7fd8e8e00640 1 -- 172.21.15.88:0/2245230486 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e40d0110 con 0x7fd8e415f270 2024-04-26T20:01:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.034+0000 7fd8e8e00640 1 --2- 172.21.15.88:0/2245230486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009870 tx=0x7fd8d402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37670499bb22dea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.035+0000 7fd8e3e00640 1 -- 172.21.15.88:0/2245230486 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4037070 con 0x7fd8e415f270 2024-04-26T20:01:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.035+0000 7fd8e3e00640 1 -- 172.21.15.88:0/2245230486 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d4032440 con 0x7fd8e415f270 2024-04-26T20:01:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.035+0000 7fd8e3e00640 1 -- 172.21.15.88:0/2245230486 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4032810 con 0x7fd8e415f270 2024-04-26T20:01:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.035+0000 7fd8ea200640 1 -- 172.21.15.88:0/2245230486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 msgr2=0x7fd8e40d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.035+0000 7fd8ea200640 1 --2- 172.21.15.88:0/2245230486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009870 tx=0x7fd8d402a250 comp rx=0 tx=0).stop 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 -- 172.21.15.88:0/2245230486 shutdown_connections 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 --2- 172.21.15.88:0/2245230486 >> v2:172.21.15.132:3300/0 conn(0x7fd8e40d3c90 0x7fd8e40d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 --2- 172.21.15.88:0/2245230486 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e415f270 0x7fd8e40d3750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 -- 172.21.15.88:0/2245230486 >> 172.21.15.88:0/2245230486 conn(0x7fd8e409e2e0 msgr2=0x7fd8e40d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 -- 172.21.15.88:0/2245230486 shutdown_connections 2024-04-26T20:01:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.036+0000 7fd8ea200640 1 -- 172.21.15.88:0/2245230486 wait complete. 2024-04-26T20:01:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.037+0000 7fd8ea200640 1 Processor -- start 2024-04-26T20:01:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.037+0000 7fd8ea200640 1 -- start start 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e40d1c20 0x7fd8e415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8e40d3c90 0x7fd8e4161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d09b0 con 0x7fd8e415f270 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e40d06d0 con 0x7fd8e40d3c90 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.038+0000 7fd8ea200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e40cfae0 con 0x7fd8e40d1c20 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46350/0 (socket says 172.21.15.88:46350) 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 -- 172.21.15.88:0/997468566 learned_addr learned my addr 172.21.15.88:0/997468566 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e8e00640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e40d1c20 0x7fd8e415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8dbe00640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8e40d3c90 0x7fd8e4161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 -- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e40d1c20 msgr2=0x7fd8e415ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e40d1c20 0x7fd8e415ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 -- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8e40d3c90 msgr2=0x7fd8e4161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8e40d3c90 0x7fd8e4161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 -- 172.21.15.88:0/997468566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e40cce10 con 0x7fd8e415f270 2024-04-26T20:01:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.039+0000 7fd8e9800640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c6b0 tx=0x7fd8dc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.040+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc003b70 con 0x7fd8e415f270 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.040+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc003d30 con 0x7fd8e415f270 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.040+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005310 con 0x7fd8e415f270 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.040+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e40cd0f0 con 0x7fd8e415f270 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.040+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e40cd6b0 con 0x7fd8e415f270 2024-04-26T20:01:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.041+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e40693c0 con 0x7fd8e415f270 2024-04-26T20:01:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.042+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8dc0054b0 con 0x7fd8e415f270 2024-04-26T20:01:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.043+0000 7fd8e1600640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 0x7fd8c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.043+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7fd8dc08b720 con 0x7fd8e415f270 2024-04-26T20:01:47.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.043+0000 7fd8e8e00640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 0x7fd8c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.044+0000 7fd8e8e00640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 0x7fd8c406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4034000 tx=0x7fd8d402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.048+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8e40d0ed0 con 0x7fd8e415f270 2024-04-26T20:01:47.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.248+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8e40cb620 con 0x7fd8e415f270 2024-04-26T20:01:47.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.249+0000 7fd8e1600640 1 -- 172.21.15.88:0/997468566 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7fd8dc059de0 con 0x7fd8e415f270 2024-04-26T20:01:47.250 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 msgr2=0x7fd8c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 0x7fd8c406edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4034000 tx=0x7fd8d402e000 comp rx=0 tx=0).stop 2024-04-26T20:01:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 msgr2=0x7fd8e4161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c6b0 tx=0x7fd8dc00cb70 comp rx=0 tx=0).stop 2024-04-26T20:01:47.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 shutdown_connections 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8e415f270 0x7fd8e4161f50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8e40d3c90 0x7fd8e4161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd8c406c8f0 0x7fd8c406edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 --2- 172.21.15.88:0/997468566 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd8e40d1c20 0x7fd8e415ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 >> 172.21.15.88:0/997468566 conn(0x7fd8e409e2e0 msgr2=0x7fd8e40ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 shutdown_connections 2024-04-26T20:01:47.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.252+0000 7fd8ea200640 1 -- 172.21.15.88:0/997468566 wait complete. 2024-04-26T20:01:47.272 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2024-04-26T20:01:47.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.467+0000 7f0010e00640 1 Processor -- start 2024-04-26T20:01:47.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.467+0000 7f0010e00640 1 -- start start 2024-04-26T20:01:47.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.469+0000 7f0010e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 0x7f000c1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.469+0000 7f0010e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f000c06c770 0x7f000c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.469+0000 7f0010e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c0b5fb0 con 0x7f000c15f740 2024-04-26T20:01:47.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.469+0000 7f0010e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f000c159c80 con 0x7f000c06c770 2024-04-26T20:01:47.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.469+0000 7f0010e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f000c06d110 con 0x7f000c1598a0 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f0009e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 0x7f000c1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f0009400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f000c06c770 0x7f000c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f000a800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f000c15f740 0x7f000c0add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:44192/0 (socket says 172.21.15.88:44192) 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f0009400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f000c06c770 0x7f000c06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:37524/0 (socket says 172.21.15.88:37524) 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f0009e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 0x7f000c1638b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:50086/0 (socket says 172.21.15.88:50086) 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.470+0000 7f0009400640 1 -- 172.21.15.88:0/3312556420 learned_addr learned my addr 172.21.15.88:0/3312556420 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.471+0000 7f0009400640 1 -- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 msgr2=0x7f000c1638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.471+0000 7f0009400640 1 --2- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 0x7f000c1638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.471+0000 7f0009400640 1 -- 172.21.15.88:0/3312556420 >> v1:172.21.15.132:6789/0 conn(0x7f000c15f740 legacy=0x7f000c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.471+0000 7f0009400640 1 -- 172.21.15.88:0/3312556420 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c160370 con 0x7f000c06c770 2024-04-26T20:01:47.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.473+0000 7f0009400640 1 --2- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c06cbd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d1bd2fc4719b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.473+0000 7f0008a00640 1 -- 172.21.15.88:0/3312556420 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4037070 con 0x7f000c06c770 2024-04-26T20:01:47.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.473+0000 7f0008a00640 1 -- 172.21.15.88:0/3312556420 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff4032440 con 0x7f000c06c770 2024-04-26T20:01:47.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.473+0000 7f0008a00640 1 -- 172.21.15.88:0/3312556420 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4032810 con 0x7f000c06c770 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.473+0000 7f0010e00640 1 -- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 msgr2=0x7f000c06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 --2- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c06cbd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 -- 172.21.15.88:0/3312556420 shutdown_connections 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 --2- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c06cbd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 --2- 172.21.15.88:0/3312556420 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c1598a0 0x7f000c1638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 -- 172.21.15.88:0/3312556420 >> 172.21.15.88:0/3312556420 conn(0x7f000c0b2d50 msgr2=0x7f000c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 -- 172.21.15.88:0/3312556420 shutdown_connections 2024-04-26T20:01:47.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.474+0000 7f0010e00640 1 -- 172.21.15.88:0/3312556420 wait complete. 2024-04-26T20:01:47.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.475+0000 7f0010e00640 1 Processor -- start 2024-04-26T20:01:47.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.475+0000 7f0010e00640 1 -- start start 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c15f740 0x7f000c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f000c163390 con 0x7f000c1598a0 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f000c1630b0 con 0x7f000c06c770 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.476+0000 7f0010e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f000c160650 con 0x7f000c15f740 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46352/0 (socket says 172.21.15.88:46352) 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 -- 172.21.15.88:0/3723926818 learned_addr learned my addr 172.21.15.88:0/3723926818 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f000a800640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c15f740 0x7f000c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 -- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c15f740 msgr2=0x7f000c147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c15f740 0x7f000c147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 -- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 msgr2=0x7f000c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:47.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.477+0000 7f0009400640 1 -- 172.21.15.88:0/3723926818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c148a30 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0009400640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4041070 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff403c870 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1e8f30 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4037040 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.478+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c0ce680 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.479+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c069540 con 0x7f000c1598a0 2024-04-26T20:01:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.480+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efff4044070 con 0x7f000c1598a0 2024-04-26T20:01:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.481+0000 7f0002000640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:47.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.481+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7efff40ac170 con 0x7f000c1598a0 2024-04-26T20:01:47.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.481+0000 7f0009e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:47.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.482+0000 7f0009e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 0x7effe806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efffc009ad0 tx=0x7efffc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.485+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efff407a8d0 con 0x7f000c1598a0 2024-04-26T20:01:47.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:47.740+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) -- 0x7f000c0cd4c0 con 0x7f000c1598a0 2024-04-26T20:01:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.400+0000 7f0002000640 1 -- 172.21.15.88:0/3723926818 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v137) ==== 136+0+0 (secure 0 0 0) 0x7f000c0cd4c0 con 0x7f000c1598a0 2024-04-26T20:01:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.403+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 msgr2=0x7effe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.403+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 0x7effe806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efffc009ad0 tx=0x7efffc008040 comp rx=0 tx=0).stop 2024-04-26T20:01:48.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.403+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 msgr2=0x7f000c0cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.403+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efff4009870 tx=0x7efff40046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 shutdown_connections 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7effe806c8f0 0x7effe806edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f000c15f740 0x7f000c147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f000c1598a0 0x7f000c0cfef0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 --2- 172.21.15.88:0/3723926818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f000c06c770 0x7f000c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 >> 172.21.15.88:0/3723926818 conn(0x7f000c0b2d50 msgr2=0x7f000c0d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 shutdown_connections 2024-04-26T20:01:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.404+0000 7f0010e00640 1 -- 172.21.15.88:0/3723926818 wait complete. 2024-04-26T20:01:48.428 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:01:48.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.631+0000 7f1f68400640 1 Processor -- start 2024-04-26T20:01:48.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.631+0000 7f1f68400640 1 -- start start 2024-04-26T20:01:48.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f68400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f60147f20 0x7f1f600b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f68400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f60073da0 0x7f1f6006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f68400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f600afed0 con 0x7f1f6006ccb0 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f68400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6006d0e0 con 0x7f1f60073da0 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f68400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600ade20 con 0x7f1f60147f20 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f65000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f60073da0 0x7f1f6006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f65000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f60073da0 0x7f1f6006c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46354/0 (socket says 172.21.15.88:46354) 2024-04-26T20:01:48.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f65000640 1 -- 172.21.15.88:0/1605643621 learned_addr learned my addr 172.21.15.88:0/1605643621 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.633+0000 7f1f65a00640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f60147f20 0x7f1f600b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.634+0000 7f1f65000640 1 -- 172.21.15.88:0/1605643621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f60147f20 msgr2=0x7f1f600b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.634+0000 7f1f65000640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f60147f20 0x7f1f600b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.634+0000 7f1f65000640 1 -- 172.21.15.88:0/1605643621 >> v1:172.21.15.132:6789/0 conn(0x7f1f6006ccb0 legacy=0x7f1f60072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.634+0000 7f1f65000640 1 -- 172.21.15.88:0/1605643621 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600ae100 con 0x7f1f60073da0 2024-04-26T20:01:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f65000640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60073da0 0x7f1f6006c770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3ce1af9d196b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:48.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f5fe00640 1 -- 172.21.15.88:0/1605643621 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c037070 con 0x7f1f60073da0 2024-04-26T20:01:48.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f5fe00640 1 -- 172.21.15.88:0/1605643621 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c032440 con 0x7f1f60073da0 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f5fe00640 1 -- 172.21.15.88:0/1605643621 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c032810 con 0x7f1f60073da0 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f68400640 1 -- 172.21.15.88:0/1605643621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60073da0 msgr2=0x7f1f6006c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.635+0000 7f1f68400640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60073da0 0x7f1f6006c770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009870 tx=0x7f1f4c02a210 comp rx=0 tx=0).stop 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 -- 172.21.15.88:0/1605643621 shutdown_connections 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60073da0 0x7f1f6006c770 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 --2- 172.21.15.88:0/1605643621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f60147f20 0x7f1f600b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 -- 172.21.15.88:0/1605643621 >> 172.21.15.88:0/1605643621 conn(0x7f1f600b2d50 msgr2=0x7f1f600b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 -- 172.21.15.88:0/1605643621 shutdown_connections 2024-04-26T20:01:48.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.636+0000 7f1f68400640 1 -- 172.21.15.88:0/1605643621 wait complete. 2024-04-26T20:01:48.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.637+0000 7f1f68400640 1 Processor -- start 2024-04-26T20:01:48.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.638+0000 7f1f68400640 1 -- start start 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.638+0000 7f1f68400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f6006ccb0 0x7f1f600cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f68400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 0x7f1f601e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f68400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f68400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600ae9a0 con 0x7f1f60147f20 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f68400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600ae6c0 con 0x7f1f60073da0 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f68400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1f60074200 con 0x7f1f6006ccb0 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46356/0 (socket says 172.21.15.88:46356) 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 -- 172.21.15.88:0/2797458178 learned_addr learned my addr 172.21.15.88:0/2797458178 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f65a00640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f6006ccb0 0x7f1f600cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f65000640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 0x7f1f601e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 -- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f6006ccb0 msgr2=0x7f1f600cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f6006ccb0 0x7f1f600cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 -- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 msgr2=0x7f1f601e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 0x7f1f601e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f66400640 1 -- 172.21.15.88:0/2797458178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f601e7b10 con 0x7f1f60147f20 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.639+0000 7f1f65000640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 0x7f1f601e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f66400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800b970 tx=0x7f1f5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58019070 con 0x7f1f60147f20 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5800cb40 con 0x7f1f60147f20 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58014440 con 0x7f1f60147f20 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601e7df0 con 0x7f1f60147f20 2024-04-26T20:01:48.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.640+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601e83b0 con 0x7f1f60147f20 2024-04-26T20:01:48.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.641+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f60069650 con 0x7f1f60147f20 2024-04-26T20:01:48.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.642+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f5801c070 con 0x7f1f60147f20 2024-04-26T20:01:48.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.643+0000 7f1f5d600640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 0x7f1f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.643+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7f1f5808b560 con 0x7f1f60147f20 2024-04-26T20:01:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.643+0000 7f1f65a00640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 0x7f1f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.644+0000 7f1f65a00640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 0x7f1f4006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f601439e0 tx=0x7f1f54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:48.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.648+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f58059c30 con 0x7f1f60147f20 2024-04-26T20:01:48.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.852+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1f600b3410 con 0x7f1f60147f20 2024-04-26T20:01:48.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.857+0000 7f1f5d600640 1 -- 172.21.15.88:0/2797458178 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f1f58059530 con 0x7f1f60147f20 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 msgr2=0x7f1f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 0x7f1f4006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1f601439e0 tx=0x7f1f54008040 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 msgr2=0x7f1f6006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800b970 tx=0x7f1f5800be30 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 shutdown_connections 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f60147f20 0x7f1f6006e220 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f60073da0 0x7f1f601e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f4006c8f0 0x7f1f4006edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 --2- 172.21.15.88:0/2797458178 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f6006ccb0 0x7f1f600cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 >> 172.21.15.88:0/2797458178 conn(0x7f1f600b2d50 msgr2=0x7f1f60072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:48.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 shutdown_connections 2024-04-26T20:01:48.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:48.860+0000 7f1f68400640 1 -- 172.21.15.88:0/2797458178 wait complete. 2024-04-26T20:01:48.880 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:01:48.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:01:48.994 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:01:49.110 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:01:49.110 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:01:49.110 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:49.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:01:49.169 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:01:49.169 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:49.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:01:49.226 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:01:49.226 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:49.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:01:49.284 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:01:49.284 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:01:49.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:01:49.341 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:01:49.341 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:01:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.549+0000 7fc235200640 1 Processor -- start 2024-04-26T20:01:49.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.549+0000 7fc235200640 1 -- start start 2024-04-26T20:01:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc235200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc2300d5990 0x7fc230163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc235200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2300b82b0 0x7fc2300b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc235200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2300d3f50 con 0x7fc23015f1b0 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc235200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc2300d36b0 con 0x7fc2300b82b0 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc235200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc2300d3990 con 0x7fc2300d5990 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc22fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2300b82b0 0x7fc2300b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc22fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc2300b82b0 0x7fc2300b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54722/0 (socket says 172.21.15.88:54722) 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.551+0000 7fc22fe00640 1 -- 172.21.15.88:0/139683232 learned_addr learned my addr 172.21.15.88:0/139683232 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.552+0000 7fc22ea00640 1 --2- 172.21.15.88:0/139683232 >> v2:172.21.15.132:3300/0 conn(0x7fc2300d5990 0x7fc230163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.552+0000 7fc22fe00640 1 -- 172.21.15.88:0/139683232 >> v2:172.21.15.132:3300/0 conn(0x7fc2300d5990 msgr2=0x7fc230163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.552+0000 7fc22fe00640 1 --2- 172.21.15.88:0/139683232 >> v2:172.21.15.132:3300/0 conn(0x7fc2300d5990 0x7fc230163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.552+0000 7fc22fe00640 1 -- 172.21.15.88:0/139683232 >> v1:172.21.15.132:6789/0 conn(0x7fc23015f1b0 legacy=0x7fc230162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.552+0000 7fc22fe00640 1 -- 172.21.15.88:0/139683232 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2300d3c70 con 0x7fc2300b82b0 2024-04-26T20:01:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.553+0000 7fc22fe00640 1 --2- 172.21.15.88:0/139683232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300b8740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc228009870 tx=0x7fc22802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eaf5fdc5cf93cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.554+0000 7fc22e000640 1 -- 172.21.15.88:0/139683232 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228037070 con 0x7fc2300b82b0 2024-04-26T20:01:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.554+0000 7fc22e000640 1 -- 172.21.15.88:0/139683232 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc228032440 con 0x7fc2300b82b0 2024-04-26T20:01:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.554+0000 7fc22e000640 1 -- 172.21.15.88:0/139683232 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228032810 con 0x7fc2300b82b0 2024-04-26T20:01:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 -- 172.21.15.88:0/139683232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 msgr2=0x7fc2300b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 --2- 172.21.15.88:0/139683232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300b8740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc228009870 tx=0x7fc22802a270 comp rx=0 tx=0).stop 2024-04-26T20:01:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 -- 172.21.15.88:0/139683232 shutdown_connections 2024-04-26T20:01:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 --2- 172.21.15.88:0/139683232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300b8740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 --2- 172.21.15.88:0/139683232 >> v2:172.21.15.132:3300/0 conn(0x7fc2300d5990 0x7fc230163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 -- 172.21.15.88:0/139683232 >> 172.21.15.88:0/139683232 conn(0x7fc23009e230 msgr2=0x7fc2300d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 -- 172.21.15.88:0/139683232 shutdown_connections 2024-04-26T20:01:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.555+0000 7fc235200640 1 -- 172.21.15.88:0/139683232 wait complete. 2024-04-26T20:01:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.557+0000 7fc235200640 1 Processor -- start 2024-04-26T20:01:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.557+0000 7fc235200640 1 -- start start 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 0x7fc2300cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 0x7fc23015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc230163750 con 0x7fc2300b82b0 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc2300d33a0 con 0x7fc23015f1b0 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.558+0000 7fc235200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc2301638d0 con 0x7fc2300d5990 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54736/0 (socket says 172.21.15.88:54736) 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 -- 172.21.15.88:0/1546522551 learned_addr learned my addr 172.21.15.88:0/1546522551 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22ea00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 0x7fc2300cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22fe00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 0x7fc23015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 -- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 msgr2=0x7fc2300cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:49.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 0x7fc2300cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 -- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 msgr2=0x7fc23015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 0x7fc23015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 -- 172.21.15.88:0/1546522551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2300cc860 con 0x7fc2300b82b0 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22ea00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 0x7fc2300cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22fe00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 0x7fc23015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:49.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.559+0000 7fc22f400640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b420 tx=0x7fc22000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.560+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220011830 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.560+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc220016070 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.560+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220011c10 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.560+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2300ccb40 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.560+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2300cd100 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.561+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2300693c0 con 0x7fc2300b82b0 2024-04-26T20:01:49.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.562+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc22001b460 con 0x7fc2300b82b0 2024-04-26T20:01:49.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.563+0000 7fc227400640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:49.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.563+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5029+0+0 (secure 0 0 0) 0x7fc22008b3f0 con 0x7fc2300b82b0 2024-04-26T20:01:49.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.563+0000 7fc22ea00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:49.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.564+0000 7fc22ea00640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 0x7fc20c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc218007330 tx=0x7fc21800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:49.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.567+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc220020020 con 0x7fc2300b82b0 2024-04-26T20:01:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.809+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2300b6be0 con 0x7fc2300b82b0 2024-04-26T20:01:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.810+0000 7fc227400640 1 -- 172.21.15.88:0/1546522551 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3872 (secure 0 0 0) 0x7fc220059ab0 con 0x7fc2300b82b0 2024-04-26T20:01:49.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:49.811 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/1772728535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":1772728535},{"type":"v1","addr":"172.21.15.132:6834","nonce":1772728535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/1799456532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1799456532},{"type":"v1","addr":"172.21.15.132:6835","nonce":1799456532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/809249379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6835","nonce":809249379},{"type":"v1","addr":"172.21.15.88:6837","nonce":809249379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6836/2803362092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2803362092},{"type":"v1","addr":"172.21.15.88:6836","nonce":2803362092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-04-26T20:01:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 22 2024-04-26T20:01:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 msgr2=0x7fc20c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 0x7fc20c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc218007330 tx=0x7fc21800b040 comp rx=0 tx=0).stop 2024-04-26T20:01:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 msgr2=0x7fc2300a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc22000b420 tx=0x7fc22000b8e0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 shutdown_connections 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc23015f1b0 0x7fc23015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc20c06c8f0 0x7fc20c06edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc2300d5990 0x7fc2300cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 --2- 172.21.15.88:0/1546522551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2300b82b0 0x7fc2300a2300 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 >> 172.21.15.88:0/1546522551 conn(0x7fc23009e230 msgr2=0x7fc2300cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.813+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 shutdown_connections 2024-04-26T20:01:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:49.814+0000 7fc235200640 1 -- 172.21.15.88:0/1546522551 wait complete. 2024-04-26T20:01:49.834 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:01:49.834 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:01:49.834 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:01:49.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:01:49.835 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:01:49.835 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:01:49.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:01:49.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:01:49.835 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:01:49.835 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:01:49.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:01:49.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:01:49.836 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:01:49.836 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:01:49.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:01:49.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:01:49.853 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:01:49.853 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:01:49.871 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:01:49.872 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:01:49.875 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:01:49.877 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:01:49.900 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:01:49.901 DEBUG:teuthology.parallel:result is None 2024-04-26T20:01:49.922 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:01:49.923 DEBUG:teuthology.parallel:result is None 2024-04-26T20:01:49.929 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:01:49.929 DEBUG:teuthology.parallel:result is None 2024-04-26T20:01:50.071 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-04-26T20:01:50.076 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-26T20:01:50.080 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-26T20:01:53.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-26T20:01:55.853 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:01:55.853 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:01:55.861 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:01:55.861 DEBUG:teuthology.parallel:result is None 2024-04-26T20:01:55.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:01:56.020 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:01:56.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.051+0000 7fe65d000640 1 Processor -- start 2024-04-26T20:01:56.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.051+0000 7fe65d000640 1 -- start start 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe65d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 0x7fe658163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe65d000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe6580b82b0 0x7fe6580b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe65d000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6580d3f50 con 0x7fe65815f1b0 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe65d000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe6580d36b0 con 0x7fe6580b82b0 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe65d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe6580d3990 con 0x7fe6580d5990 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe657400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe65815f1b0 0x7fe658162b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43902/0 (socket says 172.21.15.88:43902) 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe657400640 1 -- 172.21.15.88:0/4056111216 learned_addr learned my addr 172.21.15.88:0/4056111216 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.052+0000 7fe657e00640 1 --2- 172.21.15.88:0/4056111216 >> v2:172.21.15.88:3300/0 conn(0x7fe6580b82b0 0x7fe6580b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe656a00640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 0x7fe658163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe657e00640 1 -- 172.21.15.88:0/4056111216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 msgr2=0x7fe658163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe656000640 1 -- 172.21.15.88:0/4056111216 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311551547 0 0) 0x7fe6580d3f50 con 0x7fe65815f1b0 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe657e00640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 0x7fe658163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe657e00640 1 -- 172.21.15.88:0/4056111216 >> v1:172.21.15.88:6789/0 conn(0x7fe65815f1b0 legacy=0x7fe658162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe657e00640 1 -- 172.21.15.88:0/4056111216 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6580d3c70 con 0x7fe6580b82b0 2024-04-26T20:01:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.053+0000 7fe656a00640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 0x7fe658163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:56.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.054+0000 7fe657e00640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580b82b0 0x7fe6580b8740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe65002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e0b0fdcf862cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:56.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.054+0000 7fe656000640 1 -- 172.21.15.88:0/4056111216 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650007740 con 0x7fe6580b82b0 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.054+0000 7fe656000640 1 -- 172.21.15.88:0/4056111216 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6500078e0 con 0x7fe6580b82b0 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.054+0000 7fe656000640 1 -- 172.21.15.88:0/4056111216 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650007cb0 con 0x7fe6580b82b0 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 -- 172.21.15.88:0/4056111216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580b82b0 msgr2=0x7fe6580b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580b82b0 0x7fe6580b8740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe650009d90 tx=0x7fe65002a660 comp rx=0 tx=0).stop 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 -- 172.21.15.88:0/4056111216 shutdown_connections 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580b82b0 0x7fe6580b8740 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 --2- 172.21.15.88:0/4056111216 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe6580d5990 0x7fe658163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 -- 172.21.15.88:0/4056111216 >> 172.21.15.88:0/4056111216 conn(0x7fe65809e230 msgr2=0x7fe6580d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 -- 172.21.15.88:0/4056111216 shutdown_connections 2024-04-26T20:01:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.055+0000 7fe65d000640 1 -- 172.21.15.88:0/4056111216 wait complete. 2024-04-26T20:01:56.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.057+0000 7fe65d000640 1 Processor -- start 2024-04-26T20:01:56.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.057+0000 7fe65d000640 1 -- start start 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6580b82b0 0x7fe6580cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe65815f1b0 0x7fe65806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe658163750 con 0x7fe6580d5990 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe6580d33a0 con 0x7fe6580b82b0 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe65d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe6581638d0 con 0x7fe65815f1b0 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.058+0000 7fe656a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54800/0 (socket says 172.21.15.88:54800) 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 -- 172.21.15.88:0/1960458157 learned_addr learned my addr 172.21.15.88:0/1960458157 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe657400640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6580b82b0 0x7fe6580cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe657e00640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe65815f1b0 0x7fe65806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 -- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe65815f1b0 msgr2=0x7fe65806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:56.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe65815f1b0 0x7fe65806d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:56.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 -- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6580b82b0 msgr2=0x7fe6580cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:56.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6580b82b0 0x7fe6580cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:56.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 -- 172.21.15.88:0/1960458157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65806dd10 con 0x7fe6580d5990 2024-04-26T20:01:56.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.059+0000 7fe656a00640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c820 tx=0x7fe64400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.060+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644015070 con 0x7fe6580d5990 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.060+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe644007a30 con 0x7fe6580d5990 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.060+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644007e00 con 0x7fe6580d5990 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.060+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65815f6c0 con 0x7fe6580d5990 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.060+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65815fc80 con 0x7fe6580d5990 2024-04-26T20:01:56.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.061+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6580693c0 con 0x7fe6580d5990 2024-04-26T20:01:56.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.062+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe644018070 con 0x7fe6580d5990 2024-04-26T20:01:56.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.063+0000 7fe64f400640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 0x7fe63806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:56.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.063+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5029+0+0 (secure 0 0 0) 0x7fe64408b890 con 0x7fe6580d5990 2024-04-26T20:01:56.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.064+0000 7fe657400640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 0x7fe63806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:56.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.064+0000 7fe657400640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 0x7fe63806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe648000f60 tx=0x7fe648007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:56.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.068+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe65806dd10 con 0x7fe6580d5990 2024-04-26T20:01:56.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:56.279+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe65808d090 con 0x7fe6580d5990 2024-04-26T20:01:57.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.064+0000 7fe64f400640 1 -- 172.21.15.88:0/1960458157 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7fe644059f50 con 0x7fe6580d5990 2024-04-26T20:01:57.065 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 msgr2=0x7fe63806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 0x7fe63806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe648000f60 tx=0x7fe648007c00 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 msgr2=0x7fe658159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c820 tx=0x7fe64400cc20 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 shutdown_connections 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe63806c8f0 0x7fe63806edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe65815f1b0 0x7fe65806d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6580d5990 0x7fe658159cf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 --2- 172.21.15.88:0/1960458157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6580b82b0 0x7fe6580cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.068+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 >> 172.21.15.88:0/1960458157 conn(0x7fe65809e230 msgr2=0x7fe6580b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:57.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.069+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 shutdown_connections 2024-04-26T20:01:57.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.069+0000 7fe65d000640 1 -- 172.21.15.88:0/1960458157 wait complete. 2024-04-26T20:01:57.090 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:01:57.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.286+0000 7f43db200640 1 Processor -- start 2024-04-26T20:01:57.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.286+0000 7f43db200640 1 -- start start 2024-04-26T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.287+0000 7f43db200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f43d406fa90 0x7f43d415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.287+0000 7f43db200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.287+0000 7f43db200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d406fef0 con 0x7f43d4073720 2024-04-26T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.287+0000 7f43db200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f43d40701a0 con 0x7f43d406fa90 2024-04-26T20:01:57.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.287+0000 7f43db200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f43d406df80 con 0x7f43d406ed40 2024-04-26T20:01:57.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.288+0000 7f43da800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.288+0000 7f43da800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:60178/0 (socket says 172.21.15.88:60178) 2024-04-26T20:01:57.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.288+0000 7f43da800640 1 -- 172.21.15.88:0/1312782833 learned_addr learned my addr 172.21.15.88:0/1312782833 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:57.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.288+0000 7f43d9400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f43d406fa90 0x7f43d415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.289+0000 7f43da800640 1 -- 172.21.15.88:0/1312782833 >> v2:172.21.15.132:3300/0 conn(0x7f43d406fa90 msgr2=0x7f43d415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:57.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.289+0000 7f43da800640 1 --2- 172.21.15.88:0/1312782833 >> v2:172.21.15.132:3300/0 conn(0x7f43d406fa90 0x7f43d415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.289+0000 7f43da800640 1 -- 172.21.15.88:0/1312782833 >> v1:172.21.15.132:6789/0 conn(0x7f43d4073720 legacy=0x7f43d4073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.289+0000 7f43da800640 1 -- 172.21.15.88:0/1312782833 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d406e260 con 0x7f43d406ed40 2024-04-26T20:01:57.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.290+0000 7f43da800640 1 --2- 172.21.15.88:0/1312782833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009e50 tx=0x7f43d002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b761c5bc0a7525b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.291+0000 7f43d8a00640 1 -- 172.21.15.88:0/1312782833 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0036070 con 0x7f43d406ed40 2024-04-26T20:01:57.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.291+0000 7f43d8a00640 1 -- 172.21.15.88:0/1312782833 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d0031600 con 0x7f43d406ed40 2024-04-26T20:01:57.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.291+0000 7f43d8a00640 1 -- 172.21.15.88:0/1312782833 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d00319d0 con 0x7f43d406ed40 2024-04-26T20:01:57.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.291+0000 7f43db200640 1 -- 172.21.15.88:0/1312782833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 msgr2=0x7f43d40cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.291+0000 7f43db200640 1 --2- 172.21.15.88:0/1312782833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009e50 tx=0x7f43d002a710 comp rx=0 tx=0).stop 2024-04-26T20:01:57.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 -- 172.21.15.88:0/1312782833 shutdown_connections 2024-04-26T20:01:57.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 --2- 172.21.15.88:0/1312782833 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d406ed40 0x7f43d40cb130 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 --2- 172.21.15.88:0/1312782833 >> v2:172.21.15.132:3300/0 conn(0x7f43d406fa90 0x7f43d415dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 -- 172.21.15.88:0/1312782833 >> 172.21.15.88:0/1312782833 conn(0x7f43d40d0ae0 msgr2=0x7f43d40d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:57.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 -- 172.21.15.88:0/1312782833 shutdown_connections 2024-04-26T20:01:57.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.292+0000 7f43db200640 1 -- 172.21.15.88:0/1312782833 wait complete. 2024-04-26T20:01:57.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.293+0000 7f43db200640 1 Processor -- start 2024-04-26T20:01:57.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.294+0000 7f43db200640 1 -- start start 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.294+0000 7f43db200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.294+0000 7f43db200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d406fa90 0x7f43d415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43db200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d4073720 0x7f43d413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43db200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4146910 con 0x7f43d406ed40 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43db200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f43d406f1f0 con 0x7f43d406fa90 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43db200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f43d4146a90 con 0x7f43d4073720 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54810/0 (socket says 172.21.15.88:54810) 2024-04-26T20:01:57.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 -- 172.21.15.88:0/3748364376 learned_addr learned my addr 172.21.15.88:0/3748364376 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9400640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d406fa90 0x7f43d415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43da800640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d4073720 0x7f43d413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 -- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d4073720 msgr2=0x7f43d413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d4073720 0x7f43d413e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 -- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d406fa90 msgr2=0x7f43d415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d406fa90 0x7f43d415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.295+0000 7f43d9e00640 1 -- 172.21.15.88:0/3748364376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d415ebf0 con 0x7f43d406ed40 2024-04-26T20:01:57.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43d9e00640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80052d0 con 0x7f43d406ed40 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c8005490 con 0x7f43d406ed40 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8007260 con 0x7f43d406ed40 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d415cb50 con 0x7f43d406ed40 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.296+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d40b3db0 con 0x7f43d406ed40 2024-04-26T20:01:57.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.297+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d40693c0 con 0x7f43d406ed40 2024-04-26T20:01:57.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.298+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43c8007400 con 0x7f43d406ed40 2024-04-26T20:01:57.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.299+0000 7f43ce000640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 0x7f43b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.299+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4857+0+0 (secure 0 0 0) 0x7f43c808acd0 con 0x7f43d406ed40 2024-04-26T20:01:57.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.299+0000 7f43d9400640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 0x7f43b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.300+0000 7f43d9400640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 0x7f43b806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43d413c860 tx=0x7f43c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.304+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43c80594e0 con 0x7f43d406ed40 2024-04-26T20:01:57.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.574+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f43d4069bd0 con 0x7f43d406ed40 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.575+0000 7f43ce000640 1 -- 172.21.15.88:0/3748364376 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f43c8058de0 con 0x7f43d406ed40 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.576 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0jr1OBAAQRkT9BgmjXURkkrr7MAHWQ==", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmpzxOGBAADbar8X1eXjXWjfB60egiBQ==", 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.577 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmCnKlCxAAo0jGLXT6bRL3qpWufJJ5Sw==", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.578 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmRfxVJhAAGgPI5k+fa3OuMZs2kt0msg==", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB3ByxmU+QbKBAAKGuevMSCLF8SYrdlt/62YQ==", 2024-04-26T20:01:57.579 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ByxmJeeNMxAA64NhSfVOpwUK5gJ6cQVDvQ==", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.580 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB+Byxm0QbGAhAA6ufvhdQMCNOPOhioCEovNg==", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCBByxmpToFDxAAAuw5Ew77CrBtkai6YP0AlA==", 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.581 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCHByxmou+EHBAAkwkBnJbDkuczBKTj/szbYA==", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCKByxmNUjQNxAAz8gNq0pCsDTYh5Fo7xzwbg==", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCOByxmDaIOFxAAoLuo0jYUxlFBf/2VpwDkPw==", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.583 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCRByxmaXFiIBAAb4ALfWrAFga124RrQFb4Jw==", 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.584 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw==", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0ByxmbJEcCRAA2X7/8G8AR+PRlmM/59Ebqg==", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:01:57.585 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmEZsPOxAAJZIw3+ol9NxsWyB851hSRg==", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.586 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0Byxm+tkaEhAAXoURy4RGA/OZsj7sUhk8ZQ==", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBxByxmqnYUOBAAkUPOrzy4lbEq83+B/FNZlw==", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:01:57.587 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0kFbLxAApmvQFSMZqcrF7d0MGYbgQw==", 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:01:57.588 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0PeZJhAAmWcbnXTpRhnfZRdvKzbTTg==", 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-26T20:01:57.589 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:01:57.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 msgr2=0x7f43b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 0x7f43b806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f43d413c860 tx=0x7f43c4008040 comp rx=0 tx=0).stop 2024-04-26T20:01:57.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 msgr2=0x7f43d41549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c430 tx=0x7f43c800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 shutdown_connections 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f43b806c8f0 0x7f43b806edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f43d4073720 0x7f43d413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d406fa90 0x7f43d415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 --2- 172.21.15.88:0/3748364376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d406ed40 0x7f43d41549f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 >> 172.21.15.88:0/3748364376 conn(0x7f43d40d0ae0 msgr2=0x7f43d40ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 shutdown_connections 2024-04-26T20:01:57.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.578+0000 7f43db200640 1 -- 172.21.15.88:0/3748364376 wait complete. 2024-04-26T20:01:57.597 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:01:57.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.782+0000 7f1e0cc00640 1 Processor -- start 2024-04-26T20:01:57.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.782+0000 7f1e0cc00640 1 -- start start 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.783+0000 7f1e0cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.783+0000 7f1e0cc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e0815cb70 0x7f1e08073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e0cc00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0806d090 con 0x7f1e0815cfd0 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e0cc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1e080af460 con 0x7f1e0815cb70 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e0cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1e080ae850 con 0x7f1e0806c770 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1dffe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e0815cb70 0x7f1e08073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e06800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1e0815cfd0 0x7f1e08073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43916/0 (socket says 172.21.15.88:43916) 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e06800640 1 -- 172.21.15.88:0/3850469716 learned_addr learned my addr 172.21.15.88:0/3850469716 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e05e00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1dffe00640 1 -- 172.21.15.88:0/3850469716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 msgr2=0x7f1e0806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:57.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1dffe00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1dffe00640 1 -- 172.21.15.88:0/3850469716 >> v1:172.21.15.88:6789/0 conn(0x7f1e0815cfd0 legacy=0x7f1e08073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1dffe00640 1 -- 172.21.15.88:0/3850469716 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e080aeb30 con 0x7f1e0815cb70 2024-04-26T20:01:57.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.784+0000 7f1e05400640 1 -- 172.21.15.88:0/3850469716 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230376341 0 0) 0x7f1e0806d090 con 0x7f1e0815cfd0 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.785+0000 7f1dffe00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e08073120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1df00097a0 tx=0x7f1df002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6a82012d521fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e05400640 1 -- 172.21.15.88:0/3850469716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037070 con 0x7f1e0815cb70 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e05400640 1 -- 172.21.15.88:0/3850469716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df0032440 con 0x7f1e0815cb70 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e05400640 1 -- 172.21.15.88:0/3850469716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0032810 con 0x7f1e0815cb70 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 -- 172.21.15.88:0/3850469716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 msgr2=0x7f1e08073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e08073120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1df00097a0 tx=0x7f1df002a180 comp rx=0 tx=0).stop 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 -- 172.21.15.88:0/3850469716 shutdown_connections 2024-04-26T20:01:57.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e08073120 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/3850469716 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 -- 172.21.15.88:0/3850469716 >> 172.21.15.88:0/3850469716 conn(0x7f1e080b2d50 msgr2=0x7f1e080b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 -- 172.21.15.88:0/3850469716 shutdown_connections 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.786+0000 7f1e0cc00640 1 -- 172.21.15.88:0/3850469716 wait complete. 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.788+0000 7f1e0cc00640 1 Processor -- start 2024-04-26T20:01:57.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.788+0000 7f1e0cc00640 1 -- start start 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0815cfd0 0x7f1e0814de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0815c600 con 0x7f1e0815cb70 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1e080af0f0 con 0x7f1e0815cfd0 2024-04-26T20:01:57.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1e0cc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0815c780 con 0x7f1e0806c770 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1dffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1dffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54838/0 (socket says 172.21.15.88:54838) 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.789+0000 7f1dffe00640 1 -- 172.21.15.88:0/2825133954 learned_addr learned my addr 172.21.15.88:0/2825133954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 -- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 msgr2=0x7f1e0809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 -- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0815cfd0 msgr2=0x7f1e0814de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0815cfd0 0x7f1e0814de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:57.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 -- 172.21.15.88:0/2825133954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0815a880 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dffe00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1df00098c0 tx=0x7f1df00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037040 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df0030770 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0815ab60 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0030b40 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.790+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0815b120 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.791+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e08069540 con 0x7f1e0815cb70 2024-04-26T20:01:57.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.792+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1df0044020 con 0x7f1e0815cb70 2024-04-26T20:01:57.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.793+0000 7f1dfe000640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 0x7f1de406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:57.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.793+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4857+0+0 (secure 0 0 0) 0x7f1df00abdb0 con 0x7f1e0815cb70 2024-04-26T20:01:57.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.793+0000 7f1e05e00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 0x7f1de406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:57.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.794+0000 7f1e05e00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 0x7f1de406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1df80098f0 tx=0x7f1df8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:57.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:57.798+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1df00b0070 con 0x7f1e0815cb70 2024-04-26T20:01:58.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.000+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e080cd430 con 0x7f1e0815cb70 2024-04-26T20:01:58.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.002+0000 7f1dfe000640 1 -- 172.21.15.88:0/2825133954 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+12879 (secure 0 0 0) 0x7f1df0032e20 con 0x7f1e0815cb70 2024-04-26T20:01:58.003 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:58.004 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":48,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:01:57.056711+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 msgr2=0x7f1de406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 0x7f1de406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1df80098f0 tx=0x7f1df8008040 comp rx=0 tx=0).stop 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 msgr2=0x7f1e0809c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1df00098c0 tx=0x7f1df00046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 shutdown_connections 2024-04-26T20:01:58.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e0815cfd0 0x7f1e0814de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e0815cb70 0x7f1e0809c7d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1de406c8f0 0x7f1de406edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 --2- 172.21.15.88:0/2825133954 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e0806c770 0x7f1e0809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.005+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 >> 172.21.15.88:0/2825133954 conn(0x7f1e080b2d50 msgr2=0x7f1e080b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.006+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 shutdown_connections 2024-04-26T20:01:58.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.006+0000 7f1e0cc00640 1 -- 172.21.15.88:0/2825133954 wait complete. 2024-04-26T20:01:58.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:01:58.024 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:01:58.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.219+0000 7f68efe00640 1 Processor -- start 2024-04-26T20:01:58.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.219+0000 7f68efe00640 1 -- start start 2024-04-26T20:01:58.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.221+0000 7f68efe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f68f0153dc0 0x7f68f0161520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.221+0000 7f68efe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.221+0000 7f68efe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68f0160a60 con 0x7f68f006fd10 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.221+0000 7f68efe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f68f015dfa0 con 0x7f68f0153dc0 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.221+0000 7f68efe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f68f015e280 con 0x7f68f015d420 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.222+0000 7f68eea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f68f0153dc0 0x7f68f0161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.222+0000 7f68ee000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.222+0000 7f68ee000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:60218/0 (socket says 172.21.15.88:60218) 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.222+0000 7f68ee000640 1 -- 172.21.15.88:0/819165974 learned_addr learned my addr 172.21.15.88:0/819165974 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.223+0000 7f68ee000640 1 -- 172.21.15.88:0/819165974 >> v2:172.21.15.132:3300/0 conn(0x7f68f0153dc0 msgr2=0x7f68f0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.223+0000 7f68ee000640 1 --2- 172.21.15.88:0/819165974 >> v2:172.21.15.132:3300/0 conn(0x7f68f0153dc0 0x7f68f0161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.223+0000 7f68ee000640 1 -- 172.21.15.88:0/819165974 >> v1:172.21.15.132:6789/0 conn(0x7f68f006fd10 legacy=0x7f68f00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.223+0000 7f68ee000640 1 -- 172.21.15.88:0/819165974 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f0160d20 con 0x7f68f015d420 2024-04-26T20:01:58.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.224+0000 7f68ee000640 1 --2- 172.21.15.88:0/819165974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009870 tx=0x7f68dc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4341965400e3bbde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.225+0000 7f68ed600640 1 -- 172.21.15.88:0/819165974 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc037070 con 0x7f68f015d420 2024-04-26T20:01:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.225+0000 7f68ed600640 1 -- 172.21.15.88:0/819165974 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc032440 con 0x7f68f015d420 2024-04-26T20:01:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.225+0000 7f68ed600640 1 -- 172.21.15.88:0/819165974 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc032810 con 0x7f68f015d420 2024-04-26T20:01:58.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 -- 172.21.15.88:0/819165974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 msgr2=0x7f68f015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 --2- 172.21.15.88:0/819165974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009870 tx=0x7f68dc02a250 comp rx=0 tx=0).stop 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 -- 172.21.15.88:0/819165974 shutdown_connections 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 --2- 172.21.15.88:0/819165974 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f015d420 0x7f68f015d8b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 --2- 172.21.15.88:0/819165974 >> v2:172.21.15.132:3300/0 conn(0x7f68f0153dc0 0x7f68f0161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 -- 172.21.15.88:0/819165974 >> 172.21.15.88:0/819165974 conn(0x7f68f0075f90 msgr2=0x7f68f006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 -- 172.21.15.88:0/819165974 shutdown_connections 2024-04-26T20:01:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.226+0000 7f68efe00640 1 -- 172.21.15.88:0/819165974 wait complete. 2024-04-26T20:01:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.228+0000 7f68efe00640 1 Processor -- start 2024-04-26T20:01:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.228+0000 7f68efe00640 1 -- start start 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f006fd10 0x7f68f013cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f015d420 0x7f68f014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01541a0 con 0x7f68f0153dc0 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f68f006d0e0 con 0x7f68f015d420 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68efe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f68f0154320 con 0x7f68f006fd10 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54850/0 (socket says 172.21.15.88:54850) 2024-04-26T20:01:58.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.229+0000 7f68ee000640 1 -- 172.21.15.88:0/4064346939 learned_addr learned my addr 172.21.15.88:0/4064346939 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68eea00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f006fd10 0x7f68f013cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ef400640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f015d420 0x7f68f014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 -- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f006fd10 msgr2=0x7f68f013cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f006fd10 0x7f68f013cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 -- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f015d420 msgr2=0x7f68f014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f015d420 0x7f68f014ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 -- 172.21.15.88:0/4064346939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f014b050 con 0x7f68f0153dc0 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68ee000640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68dc02a290 tx=0x7f68dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.230+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc041030 con 0x7f68f0153dc0 2024-04-26T20:01:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.231+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc030440 con 0x7f68f0153dc0 2024-04-26T20:01:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.231+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc037040 con 0x7f68f0153dc0 2024-04-26T20:01:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.231+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f0154ef0 con 0x7f68f0153dc0 2024-04-26T20:01:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.231+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01554b0 con 0x7f68f0153dc0 2024-04-26T20:01:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.232+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f0069ad0 con 0x7f68f0153dc0 2024-04-26T20:01:58.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.233+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68dc044070 con 0x7f68f0153dc0 2024-04-26T20:01:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.234+0000 7f68e6a00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 0x7f68d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.234+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4857+0+0 (secure 0 0 0) 0x7f68dc0abc70 con 0x7f68f0153dc0 2024-04-26T20:01:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.234+0000 7f68eea00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 0x7f68d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.235+0000 7f68eea00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 0x7f68d006ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ad0 tx=0x7f68e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.238+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f68dc04b070 con 0x7f68f0153dc0 2024-04-26T20:01:58.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.436+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68f014b220 con 0x7f68f0153dc0 2024-04-26T20:01:58.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.437+0000 7f68e6a00640 1 -- 172.21.15.88:0/4064346939 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f68dc07a350 con 0x7f68f0153dc0 2024-04-26T20:01:58.438 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:01:58.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 msgr2=0x7f68d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 0x7f68d006ede0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009ad0 tx=0x7f68e0008040 comp rx=0 tx=0).stop 2024-04-26T20:01:58.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 msgr2=0x7f68f01544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68dc02a290 tx=0x7f68dc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 shutdown_connections 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f015d420 0x7f68f014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f0153dc0 0x7f68f01544a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f68d006c920 0x7f68d006ede0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.439+0000 7f68efe00640 1 --2- 172.21.15.88:0/4064346939 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f68f006fd10 0x7f68f013cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.440+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 >> 172.21.15.88:0/4064346939 conn(0x7f68f0075f90 msgr2=0x7f68f006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.440+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 shutdown_connections 2024-04-26T20:01:58.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.440+0000 7f68efe00640 1 -- 172.21.15.88:0/4064346939 wait complete. 2024-04-26T20:01:58.460 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:01:58.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.651+0000 7f287f200640 1 Processor -- start 2024-04-26T20:01:58.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.651+0000 7f287f200640 1 -- start start 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.652+0000 7f287f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2880073720 0x7f288006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287f200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f288006ed40 0x7f28800cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287f200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28800701a0 con 0x7f288015dec0 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287f200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f288015e320 con 0x7f288006ed40 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f288006df80 con 0x7f2880073720 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f288006ed40 0x7f28800cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f288006ed40 0x7f28800cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54860/0 (socket says 172.21.15.88:54860) 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 -- 172.21.15.88:0/1659830038 learned_addr learned my addr 172.21.15.88:0/1659830038 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287d400640 1 --1- 172.21.15.88:0/1659830038 >> v1:172.21.15.88:6789/0 conn(0x7f288015dec0 0x7f2880073b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43920/0 (socket says 172.21.15.88:43920) 2024-04-26T20:01:58.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287de00640 1 --2- 172.21.15.88:0/1659830038 >> v2:172.21.15.132:3300/0 conn(0x7f2880073720 0x7f288006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 -- 172.21.15.88:0/1659830038 >> v2:172.21.15.132:3300/0 conn(0x7f2880073720 msgr2=0x7f288006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 --2- 172.21.15.88:0/1659830038 >> v2:172.21.15.132:3300/0 conn(0x7f2880073720 0x7f288006fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 -- 172.21.15.88:0/1659830038 >> v1:172.21.15.88:6789/0 conn(0x7f288015dec0 legacy=0x7f2880073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.653+0000 7f287e800640 1 -- 172.21.15.88:0/1659830038 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288006e260 con 0x7f288006ed40 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287e800640 1 --2- 172.21.15.88:0/1659830038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288006ed40 0x7f28800cb130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2880050f80 tx=0x7f287802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18bca605afcb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287ca00640 1 -- 172.21.15.88:0/1659830038 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878036070 con 0x7f288006ed40 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287ca00640 1 -- 172.21.15.88:0/1659830038 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2878031600 con 0x7f288006ed40 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287ca00640 1 -- 172.21.15.88:0/1659830038 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28780319d0 con 0x7f288006ed40 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287f200640 1 -- 172.21.15.88:0/1659830038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288006ed40 msgr2=0x7f28800cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.655+0000 7f287f200640 1 --2- 172.21.15.88:0/1659830038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288006ed40 0x7f28800cb130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2880050f80 tx=0x7f287802a730 comp rx=0 tx=0).stop 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 -- 172.21.15.88:0/1659830038 shutdown_connections 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 --2- 172.21.15.88:0/1659830038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288006ed40 0x7f28800cb130 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 --2- 172.21.15.88:0/1659830038 >> v2:172.21.15.132:3300/0 conn(0x7f2880073720 0x7f288006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 -- 172.21.15.88:0/1659830038 >> 172.21.15.88:0/1659830038 conn(0x7f28800d0ae0 msgr2=0x7f28800d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 -- 172.21.15.88:0/1659830038 shutdown_connections 2024-04-26T20:01:58.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.656+0000 7f287f200640 1 -- 172.21.15.88:0/1659830038 wait complete. 2024-04-26T20:01:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.657+0000 7f287f200640 1 Processor -- start 2024-04-26T20:01:58.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.658+0000 7f287f200640 1 -- start start 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.658+0000 7f287f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 0x7f288013b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.658+0000 7f287f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2880073720 0x7f28800aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2880146910 con 0x7f288015dec0 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f288006f1f0 con 0x7f288006ed40 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2880146a90 con 0x7f2880073720 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54872/0 (socket says 172.21.15.88:54872) 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 -- 172.21.15.88:0/2449792658 learned_addr learned my addr 172.21.15.88:0/2449792658 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287d400640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2880073720 0x7f28800aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287de00640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 0x7f288013b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 -- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2880073720 msgr2=0x7f28800aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2880073720 0x7f28800aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 -- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 msgr2=0x7f288013b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 0x7f288013b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287e800640 1 -- 172.21.15.88:0/2449792658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28800cf9b0 con 0x7f288015dec0 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.659+0000 7f287de00640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 0x7f288013b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:58.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f287e800640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f287802ad20 tx=0x7f2878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878041020 con 0x7f288015dec0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28780077a0 con 0x7f288015dec0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2878036040 con 0x7f288015dec0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28800cfc90 con 0x7f288015dec0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.660+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28800d0250 con 0x7f288015dec0 2024-04-26T20:01:58.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.661+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28800d0450 con 0x7f288015dec0 2024-04-26T20:01:58.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.662+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2878044020 con 0x7f288015dec0 2024-04-26T20:01:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.663+0000 7f2876000640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 0x7f286006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.663+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4857+0+0 (secure 0 0 0) 0x7f28780ac490 con 0x7f288015dec0 2024-04-26T20:01:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.663+0000 7f287de00640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 0x7f286006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:58.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.664+0000 7f287de00640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 0x7f286006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2870006fd0 tx=0x7f2870008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:58.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.668+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f287804b030 con 0x7f288015dec0 2024-04-26T20:01:58.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.853+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f28800693c0 con 0x7f288015dec0 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.854+0000 7f2876000640 1 -- 172.21.15.88:0/2449792658 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) ==== 84+0+65 (secure 0 0 0) 0x7f287807acd0 con 0x7f288015dec0 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:01:58.855 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:01:58.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 msgr2=0x7f286006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 0x7f286006edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2870006fd0 tx=0x7f2870008040 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 msgr2=0x7f28800c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f287802ad20 tx=0x7f2878004a40 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 shutdown_connections 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288015dec0 0x7f28800c9e60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f286006c8f0 0x7f286006edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2880073720 0x7f28800aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 --2- 172.21.15.88:0/2449792658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288006ed40 0x7f288013b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 >> 172.21.15.88:0/2449792658 conn(0x7f28800d0ae0 msgr2=0x7f28800cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 shutdown_connections 2024-04-26T20:01:58.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:58.857+0000 7f287f200640 1 -- 172.21.15.88:0/2449792658 wait complete. 2024-04-26T20:01:58.875 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:01:58.875 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:01:59.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.078+0000 7fc0ff200640 1 Processor -- start 2024-04-26T20:01:59.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.079+0000 7fc0ff200640 1 -- start start 2024-04-26T20:01:59.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0ff200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc10015f1c0 0x7fc1000d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0ff200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1000d2e60 con 0x7fc1000d0760 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0ff200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc1000d3a70 con 0x7fc10015f1c0 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0ff200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000d3140 con 0x7fc1000d3f00 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.080+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0fde00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc10015f1c0 0x7fc1000d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:34078/0 (socket says 172.21.15.88:34078) 2024-04-26T20:01:59.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 -- 172.21.15.88:0/2161683046 learned_addr learned my addr 172.21.15.88:0/2161683046 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:59.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 -- 172.21.15.88:0/2161683046 >> v2:172.21.15.132:3300/0 conn(0x7fc10015f1c0 msgr2=0x7fc1000d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:01:59.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 --2- 172.21.15.88:0/2161683046 >> v2:172.21.15.132:3300/0 conn(0x7fc10015f1c0 0x7fc1000d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:59.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 -- 172.21.15.88:0/2161683046 >> v1:172.21.15.132:6789/0 conn(0x7fc1000d0760 legacy=0x7fc100162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:59.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.081+0000 7fc0f7e00640 1 -- 172.21.15.88:0/2161683046 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000d3420 con 0x7fc1000d3f00 2024-04-26T20:01:59.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.083+0000 7fc0f7e00640 1 --2- 172.21.15.88:0/2161683046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009870 tx=0x7fc0e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6229de564ce1f3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.086+0000 7fc0fd400640 1 -- 172.21.15.88:0/2161683046 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8037070 con 0x7fc1000d3f00 2024-04-26T20:01:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.086+0000 7fc0fd400640 1 -- 172.21.15.88:0/2161683046 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e8032440 con 0x7fc1000d3f00 2024-04-26T20:01:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.086+0000 7fc0fd400640 1 -- 172.21.15.88:0/2161683046 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8032810 con 0x7fc1000d3f00 2024-04-26T20:01:59.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 -- 172.21.15.88:0/2161683046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 msgr2=0x7fc1000d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:59.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 --2- 172.21.15.88:0/2161683046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009870 tx=0x7fc0e802a270 comp rx=0 tx=0).stop 2024-04-26T20:01:59.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 -- 172.21.15.88:0/2161683046 shutdown_connections 2024-04-26T20:01:59.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 --2- 172.21.15.88:0/2161683046 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000d1f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:59.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 --2- 172.21.15.88:0/2161683046 >> v2:172.21.15.132:3300/0 conn(0x7fc10015f1c0 0x7fc1000d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:59.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 -- 172.21.15.88:0/2161683046 >> 172.21.15.88:0/2161683046 conn(0x7fc10009e340 msgr2=0x7fc1000d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:01:59.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 -- 172.21.15.88:0/2161683046 shutdown_connections 2024-04-26T20:01:59.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.087+0000 7fc0ff200640 1 -- 172.21.15.88:0/2161683046 wait complete. 2024-04-26T20:01:59.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.089+0000 7fc0ff200640 1 Processor -- start 2024-04-26T20:01:59.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.089+0000 7fc0ff200640 1 -- start start 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 0x7fc100152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc10015f720 con 0x7fc10015f1c0 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc10015f5a0 con 0x7fc1000d0760 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0ff200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000d0bf0 con 0x7fc1000d3f00 2024-04-26T20:01:59.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.090+0000 7fc0fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:59.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57056/0 (socket says 172.21.15.88:57056) 2024-04-26T20:01:59.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 -- 172.21.15.88:0/1981566703 learned_addr learned my addr 172.21.15.88:0/1981566703 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:01:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fde00640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 0x7fc100152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 -- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 msgr2=0x7fc1000b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 -- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 msgr2=0x7fc100152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 0x7fc100152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 -- 172.21.15.88:0/1981566703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000cc3f0 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fde00640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 0x7fc100152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.091+0000 7fc0fe800640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c7d0 tx=0x7fc0f800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.092+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003b70 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.092+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8003d30 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.092+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800aa90 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.092+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1000cc6d0 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.092+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001531a0 con 0x7fc10015f1c0 2024-04-26T20:01:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.093+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1000693c0 con 0x7fc10015f1c0 2024-04-26T20:01:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.094+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0f800ac30 con 0x7fc10015f1c0 2024-04-26T20:01:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.095+0000 7fc0f6000640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:01:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.095+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4857+0+0 (secure 0 0 0) 0x7fc0f808b800 con 0x7fc10015f1c0 2024-04-26T20:01:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.095+0000 7fc0fde00640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:01:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.096+0000 7fc0fde00640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009a20 tx=0x7fc0f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:01:59.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.099+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0f8059890 con 0x7fc10015f1c0 2024-04-26T20:01:59.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:01:59.298+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc10006d250 con 0x7fc10015f1c0 2024-04-26T20:02:00.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.095+0000 7fc0f6000640 1 -- 172.21.15.88:0/1981566703 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) ==== 138+0+0 (secure 0 0 0) 0x7fc0f805db90 con 0x7fc10015f1c0 2024-04-26T20:02:00.096 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-04-26T20:02:00.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.098+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 msgr2=0x7fc0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:00.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.098+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0009a20 tx=0x7fc0f0008040 comp rx=0 tx=0).stop 2024-04-26T20:02:00.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.098+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 msgr2=0x7fc100160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.098+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c7d0 tx=0x7fc0f800cc90 comp rx=0 tx=0).stop 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 shutdown_connections 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc10015f1c0 0x7fc100160d70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc1000d3f00 0x7fc1000b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 --2- 172.21.15.88:0/1981566703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1000d0760 0x7fc100152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 >> 172.21.15.88:0/1981566703 conn(0x7fc10009e340 msgr2=0x7fc10015dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:00.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 shutdown_connections 2024-04-26T20:02:00.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.099+0000 7fc0ff200640 1 -- 172.21.15.88:0/1981566703 wait complete. 2024-04-26T20:02:00.127 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:02:00.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.325+0000 7fdecbe00640 1 Processor -- start 2024-04-26T20:02:00.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.325+0000 7fdecbe00640 1 -- start start 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdecbe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdec406f450 0x7fdec40b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdecbe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 0x7fdec40b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdecbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec40b5180 con 0x7fdec415f710 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdecbe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdec41602f0 con 0x7fdec406f450 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdecbe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fdec41605d0 con 0x7fdec40b5560 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdec9400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdec406f450 0x7fdec40b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdec9400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdec406f450 0x7fdec40b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57072/0 (socket says 172.21.15.88:57072) 2024-04-26T20:02:00.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.327+0000 7fdec9400640 1 -- 172.21.15.88:0/2514153434 learned_addr learned my addr 172.21.15.88:0/2514153434 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec8a00640 1 --2- 172.21.15.88:0/2514153434 >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 0x7fdec40b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec9400640 1 -- 172.21.15.88:0/2514153434 >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 msgr2=0x7fdec40b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec9400640 1 --2- 172.21.15.88:0/2514153434 >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 0x7fdec40b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec9400640 1 -- 172.21.15.88:0/2514153434 >> v1:172.21.15.132:6789/0 conn(0x7fdec415f710 legacy=0x7fdec406dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec9400640 1 -- 172.21.15.88:0/2514153434 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec40b4160 con 0x7fdec406f450 2024-04-26T20:02:00.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.328+0000 7fdec8a00640 1 --2- 172.21.15.88:0/2514153434 >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 0x7fdec40b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:00.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.329+0000 7fdec9400640 1 --2- 172.21.15.88:0/2514153434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec406f450 0x7fdec40b4c40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdec40d0cd0 tx=0x7fdeb802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4b53031942e351 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:00.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.329+0000 7fdec3e00640 1 -- 172.21.15.88:0/2514153434 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8037070 con 0x7fdec406f450 2024-04-26T20:02:00.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.329+0000 7fdec3e00640 1 -- 172.21.15.88:0/2514153434 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb8032440 con 0x7fdec406f450 2024-04-26T20:02:00.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.329+0000 7fdec3e00640 1 -- 172.21.15.88:0/2514153434 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8032810 con 0x7fdec406f450 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 -- 172.21.15.88:0/2514153434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec406f450 msgr2=0x7fdec40b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2514153434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec406f450 0x7fdec40b4c40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdec40d0cd0 tx=0x7fdeb802a230 comp rx=0 tx=0).stop 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 -- 172.21.15.88:0/2514153434 shutdown_connections 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2514153434 >> v2:172.21.15.132:3300/0 conn(0x7fdec40b5560 0x7fdec40b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2514153434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec406f450 0x7fdec40b4c40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 -- 172.21.15.88:0/2514153434 >> 172.21.15.88:0/2514153434 conn(0x7fdec40b2a30 msgr2=0x7fdec40b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 -- 172.21.15.88:0/2514153434 shutdown_connections 2024-04-26T20:02:00.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.330+0000 7fdecbe00640 1 -- 172.21.15.88:0/2514153434 wait complete. 2024-04-26T20:02:00.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.332+0000 7fdecbe00640 1 Processor -- start 2024-04-26T20:02:00.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.333+0000 7fdecbe00640 1 -- start start 2024-04-26T20:02:00.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdec406f450 0x7fdec41511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdec415f710 0x7fdec415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdec415fbc0 con 0x7fdec40b5560 2024-04-26T20:02:00.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4074160 con 0x7fdec406f450 2024-04-26T20:02:00.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdecbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdec415fd40 con 0x7fdec415f710 2024-04-26T20:02:00.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdec8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.334+0000 7fdec8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57076/0 (socket says 172.21.15.88:57076) 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 -- 172.21.15.88:0/2014817964 learned_addr learned my addr 172.21.15.88:0/2014817964 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec9e00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdec415f710 0x7fdec415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec9400640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdec406f450 0x7fdec41511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 -- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdec415f710 msgr2=0x7fdec415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdec415f710 0x7fdec415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 -- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdec406f450 msgr2=0x7fdec41511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdec406f450 0x7fdec41511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 -- 172.21.15.88:0/2014817964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec41f52f0 con 0x7fdec40b5560 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.335+0000 7fdec8a00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c7f0 tx=0x7fdeb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.336+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0004230 con 0x7fdec40b5560 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.336+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb00043d0 con 0x7fdec40b5560 2024-04-26T20:02:00.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.336+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0007a70 con 0x7fdec40b5560 2024-04-26T20:02:00.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.336+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec4154060 con 0x7fdec40b5560 2024-04-26T20:02:00.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.336+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec415bad0 con 0x7fdec40b5560 2024-04-26T20:02:00.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.337+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec4069650 con 0x7fdec40b5560 2024-04-26T20:02:00.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.338+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeb0007c10 con 0x7fdec40b5560 2024-04-26T20:02:00.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.339+0000 7fdec1600640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 0x7fdea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:00.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.340+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5227+0+0 (secure 0 0 0) 0x7fdeb008bcf0 con 0x7fdec40b5560 2024-04-26T20:02:00.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.340+0000 7fdec9400640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 0x7fdea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:00.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.340+0000 7fdec9400640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 0x7fdea406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8034000 tx=0x7fdeb802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:00.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.343+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdec4154060 con 0x7fdec40b5560 2024-04-26T20:02:00.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-26T20:02:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:00.549+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdec40703b0 con 0x7fdec40b5560 2024-04-26T20:02:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.111+0000 7fdec1600640 1 -- 172.21.15.88:0/2014817964 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) ==== 170+0+0 (secure 0 0 0) 0x7fdeb0015020 con 0x7fdec40b5560 2024-04-26T20:02:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.114+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 msgr2=0x7fdea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.114+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 0x7fdea406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8034000 tx=0x7fdeb802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 msgr2=0x7fdec415cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c7f0 tx=0x7fdeb000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 shutdown_connections 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdea406c8f0 0x7fdea406edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdec415f710 0x7fdec415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdec40b5560 0x7fdec415cb70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 --2- 172.21.15.88:0/2014817964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdec406f450 0x7fdec41511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 >> 172.21.15.88:0/2014817964 conn(0x7fdec40b2a30 msgr2=0x7fdec4161f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 shutdown_connections 2024-04-26T20:02:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.115+0000 7fdecbe00640 1 -- 172.21.15.88:0/2014817964 wait complete. 2024-04-26T20:02:01.152 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:02:01.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.325+0000 7f9068a00640 1 Processor -- start 2024-04-26T20:02:01.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.325+0000 7f9068a00640 1 -- start start 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9068a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9064147f20 0x7f90640b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9068a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9064073da0 0x7f906406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9068a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90640afed0 con 0x7f906406ccb0 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9068a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f906406d090 con 0x7f9064073da0 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9068a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f90640ade20 con 0x7f9064147f20 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9061400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9064073da0 0x7f906406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9061400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f9064073da0 0x7f906406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57080/0 (socket says 172.21.15.88:57080) 2024-04-26T20:02:01.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9061400640 1 -- 172.21.15.88:0/957827211 learned_addr learned my addr 172.21.15.88:0/957827211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:01.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.327+0000 7f9062800640 1 --1- 172.21.15.88:0/957827211 >> v1:172.21.15.88:6789/0 conn(0x7f906406ccb0 0x7f9064072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36482/0 (socket says 172.21.15.88:36482) 2024-04-26T20:02:01.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9061e00640 1 --2- 172.21.15.88:0/957827211 >> v2:172.21.15.132:3300/0 conn(0x7f9064147f20 0x7f90640b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9061400640 1 -- 172.21.15.88:0/957827211 >> v2:172.21.15.132:3300/0 conn(0x7f9064147f20 msgr2=0x7f90640b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9061400640 1 --2- 172.21.15.88:0/957827211 >> v2:172.21.15.132:3300/0 conn(0x7f9064147f20 0x7f90640b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9061400640 1 -- 172.21.15.88:0/957827211 >> v1:172.21.15.88:6789/0 conn(0x7f906406ccb0 legacy=0x7f9064072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9061400640 1 -- 172.21.15.88:0/957827211 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90640ae100 con 0x7f9064073da0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.328+0000 7f9060a00640 1 -- 172.21.15.88:0/957827211 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 225015185 0 0) 0x7f905c001110 con 0x7f906406ccb0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.329+0000 7f9061400640 1 --2- 172.21.15.88:0/957827211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064073da0 0x7f906406c770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9064075d20 tx=0x7f904c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f90348a6a937828 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.329+0000 7f9060a00640 1 -- 172.21.15.88:0/957827211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c036070 con 0x7f9064073da0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.329+0000 7f9060a00640 1 -- 172.21.15.88:0/957827211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c031600 con 0x7f9064073da0 2024-04-26T20:02:01.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.329+0000 7f9060a00640 1 -- 172.21.15.88:0/957827211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0319d0 con 0x7f9064073da0 2024-04-26T20:02:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 -- 172.21.15.88:0/957827211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064073da0 msgr2=0x7f906406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 --2- 172.21.15.88:0/957827211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064073da0 0x7f906406c770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9064075d20 tx=0x7f904c02a730 comp rx=0 tx=0).stop 2024-04-26T20:02:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 -- 172.21.15.88:0/957827211 shutdown_connections 2024-04-26T20:02:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 --2- 172.21.15.88:0/957827211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064073da0 0x7f906406c770 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 --2- 172.21.15.88:0/957827211 >> v2:172.21.15.132:3300/0 conn(0x7f9064147f20 0x7f90640b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 -- 172.21.15.88:0/957827211 >> 172.21.15.88:0/957827211 conn(0x7f90640b2d50 msgr2=0x7f90640b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 -- 172.21.15.88:0/957827211 shutdown_connections 2024-04-26T20:02:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.330+0000 7f9068a00640 1 -- 172.21.15.88:0/957827211 wait complete. 2024-04-26T20:02:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.332+0000 7f9068a00640 1 Processor -- start 2024-04-26T20:02:01.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.332+0000 7f9068a00640 1 -- start start 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 0x7f90640d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073da0 0x7f906409cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f90640ae9a0 con 0x7f9064147f20 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f90640ae6c0 con 0x7f9064073da0 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9068a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9064074230 con 0x7f906406ccb0 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9062800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9062800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57090/0 (socket says 172.21.15.88:57090) 2024-04-26T20:02:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.333+0000 7f9062800640 1 -- 172.21.15.88:0/3857633470 learned_addr learned my addr 172.21.15.88:0/3857633470 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9061400640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073da0 0x7f906409cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9061e00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 0x7f90640d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 -- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 msgr2=0x7f90640d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 0x7f90640d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 -- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073da0 msgr2=0x7f906409cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073da0 0x7f906409cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 -- 172.21.15.88:0/3857633470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906409d470 con 0x7f9064147f20 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9061e00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 0x7f90640d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9062800640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c410 tx=0x7f905c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0074e0 con 0x7f9064147f20 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c0047e0 con 0x7f9064147f20 2024-04-26T20:02:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906409d750 con 0x7f9064147f20 2024-04-26T20:02:01.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.334+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c004bd0 con 0x7f9064147f20 2024-04-26T20:02:01.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.335+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906409dff0 con 0x7f9064147f20 2024-04-26T20:02:01.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.336+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906409db40 con 0x7f9064147f20 2024-04-26T20:02:01.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.337+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f905c011070 con 0x7f9064147f20 2024-04-26T20:02:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.338+0000 7f905a000640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 0x7f904006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:01.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.338+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5609+0+0 (secure 0 0 0) 0x7f905c08b6e0 con 0x7f9064147f20 2024-04-26T20:02:01.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.338+0000 7f9061e00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 0x7f904006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:01.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.339+0000 7f9061e00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 0x7f904006edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:01.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.343+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90640afed0 con 0x7f9064147f20 2024-04-26T20:02:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:01.582+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9064069650 con 0x7f9064147f20 2024-04-26T20:02:02.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.115+0000 7f905a000640 1 -- 172.21.15.88:0/3857633470 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v25) ==== 172+0+0 (secure 0 0 0) 0x7f905c01a2b0 con 0x7f9064147f20 2024-04-26T20:02:02.116 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 8 and data pool 9 2024-04-26T20:02:02.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 msgr2=0x7f904006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 0x7f904006edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-04-26T20:02:02.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 msgr2=0x7f9064160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c410 tx=0x7f905c00c8d0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 shutdown_connections 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9064147f20 0x7f9064160ae0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073da0 0x7f906409cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f904006c8f0 0x7f904006edb0 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9054006fd0 tx=0x7f9054008040 comp rx=0 tx=0).stop 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 --2- 172.21.15.88:0/3857633470 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f906406ccb0 0x7f90640d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.119+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 >> 172.21.15.88:0/3857633470 conn(0x7f90640b2d50 msgr2=0x7f90640b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.120+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 shutdown_connections 2024-04-26T20:02:02.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.120+0000 7f9068a00640 1 -- 172.21.15.88:0/3857633470 wait complete. 2024-04-26T20:02:02.159 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.332+0000 7fa361000640 1 Processor -- start 2024-04-26T20:02:02.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.332+0000 7fa361000640 1 -- start start 2024-04-26T20:02:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.333+0000 7fa361000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 0x7fa35c1638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.333+0000 7fa361000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa35c15f740 0x7fa35c15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.333+0000 7fa361000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa35c0b5fb0 con 0x7fa35c1598a0 2024-04-26T20:02:02.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.333+0000 7fa361000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa35c159c80 con 0x7fa35c15f740 2024-04-26T20:02:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.333+0000 7fa361000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c160110 con 0x7fa35c06ce80 2024-04-26T20:02:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa35c15f740 0x7fa35c15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa35c15f740 0x7fa35c15fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57100/0 (socket says 172.21.15.88:57100) 2024-04-26T20:02:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 -- 172.21.15.88:0/2849665321 learned_addr learned my addr 172.21.15.88:0/2849665321 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:02.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa359400640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 0x7fa35c1638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 -- 172.21.15.88:0/2849665321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 msgr2=0x7fa35c1638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 0x7fa35c1638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 -- 172.21.15.88:0/2849665321 >> v1:172.21.15.132:6789/0 conn(0x7fa35c1598a0 legacy=0x7fa35c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa35a800640 1 -- 172.21.15.88:0/2849665321 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c160370 con 0x7fa35c15f740 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.334+0000 7fa359400640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 0x7fa35c1638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:02.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.335+0000 7fa35a800640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c15f740 0x7fa35c15fbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e80 tx=0x7fa35402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caafa65878e8bf7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa358a00640 1 -- 172.21.15.88:0/2849665321 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354036070 con 0x7fa35c15f740 2024-04-26T20:02:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa358a00640 1 -- 172.21.15.88:0/2849665321 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa354031600 con 0x7fa35c15f740 2024-04-26T20:02:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa358a00640 1 -- 172.21.15.88:0/2849665321 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540319d0 con 0x7fa35c15f740 2024-04-26T20:02:02.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 -- 172.21.15.88:0/2849665321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c15f740 msgr2=0x7fa35c15fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c15f740 0x7fa35c15fbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa354009e80 tx=0x7fa35402a660 comp rx=0 tx=0).stop 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 -- 172.21.15.88:0/2849665321 shutdown_connections 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c15f740 0x7fa35c15fbd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 --2- 172.21.15.88:0/2849665321 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c06ce80 0x7fa35c1638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 -- 172.21.15.88:0/2849665321 >> 172.21.15.88:0/2849665321 conn(0x7fa35c0b2d50 msgr2=0x7fa35c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 -- 172.21.15.88:0/2849665321 shutdown_connections 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.336+0000 7fa361000640 1 -- 172.21.15.88:0/2849665321 wait complete. 2024-04-26T20:02:02.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.338+0000 7fa361000640 1 Processor -- start 2024-04-26T20:02:02.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.338+0000 7fa361000640 1 -- start start 2024-04-26T20:02:02.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c1598a0 0x7fa35c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c15f740 0x7fa35c1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c163390 con 0x7fa35c06ce80 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1630b0 con 0x7fa35c15f740 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa361000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c160650 con 0x7fa35c1598a0 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.339+0000 7fa359e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57104/0 (socket says 172.21.15.88:57104) 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 -- 172.21.15.88:0/3721547361 learned_addr learned my addr 172.21.15.88:0/3721547361 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359400640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c1598a0 0x7fa35c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 -- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c1598a0 msgr2=0x7fa35c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c1598a0 0x7fa35c0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 -- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c15f740 msgr2=0x7fa35c1f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c15f740 0x7fa35c1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 -- 172.21.15.88:0/3721547361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c0ca4f0 con 0x7fa35c06ce80 2024-04-26T20:02:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa359e00640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c460 tx=0x7fa34c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.340+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c016020 con 0x7fa35c06ce80 2024-04-26T20:02:02.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.341+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c012870 con 0x7fa35c06ce80 2024-04-26T20:02:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.341+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c012c60 con 0x7fa35c06ce80 2024-04-26T20:02:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.341+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c0cffa0 con 0x7fa35c06ce80 2024-04-26T20:02:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.341+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c0cf4c0 con 0x7fa35c06ce80 2024-04-26T20:02:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.342+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c069540 con 0x7fa35c06ce80 2024-04-26T20:02:02.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.343+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34c019020 con 0x7fa35c06ce80 2024-04-26T20:02:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.344+0000 7fa352000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.344+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5709+0+0 (secure 0 0 0) 0x7fa34c08b8a0 con 0x7fa35c06ce80 2024-04-26T20:02:02.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.344+0000 7fa359400640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.345+0000 7fa359400640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0b4310 tx=0x7fa348008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.348+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa34c020030 con 0x7fa35c06ce80 2024-04-26T20:02:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.533+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa35c073fc0 con 0x7fa35c06ce80 2024-04-26T20:02:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.535+0000 7fa352000640 1 -- 172.21.15.88:0/3721547361 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16666 (secure 0 0 0) 0x7fa34c059cc0 con 0x7fa35c06ce80 2024-04-26T20:02:02.537 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:02.537 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":51,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:01.583645+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 msgr2=0x7fa33c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 0x7fa33c06edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0b4310 tx=0x7fa348008040 comp rx=0 tx=0).stop 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 msgr2=0x7fa35c13a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00c460 tx=0x7fa34c00c920 comp rx=0 tx=0).stop 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 shutdown_connections 2024-04-26T20:02:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa35c15f740 0x7fa35c1f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa33c06c8f0 0x7fa33c06edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa35c1598a0 0x7fa35c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 --2- 172.21.15.88:0/3721547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa35c06ce80 0x7fa35c13a210 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 >> 172.21.15.88:0/3721547361 conn(0x7fa35c0b2d50 msgr2=0x7fa35c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.538+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 shutdown_connections 2024-04-26T20:02:02.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.539+0000 7fa361000640 1 -- 172.21.15.88:0/3721547361 wait complete. 2024-04-26T20:02:02.558 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.752+0000 7f8660c00640 1 Processor -- start 2024-04-26T20:02:02.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.752+0000 7f8660c00640 1 -- start start 2024-04-26T20:02:02.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8660c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f865c0d3680 0x7f865c0d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8660c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0c9520 0x7f865c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8660c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865c15f800 con 0x7f865c15f420 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8660c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f865c0d2c70 con 0x7f865c0d3680 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8660c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d3480 con 0x7f865c0c9520 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8653e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f865c0d3680 0x7f865c0d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8653e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f865c0d3680 0x7f865c0d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57120/0 (socket says 172.21.15.88:57120) 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f8653e00640 1 -- 172.21.15.88:0/195317129 learned_addr learned my addr 172.21.15.88:0/195317129 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:02.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.754+0000 7f865a800640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0c9520 0x7f865c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.755+0000 7f8653e00640 1 -- 172.21.15.88:0/195317129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0c9520 msgr2=0x7f865c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.755+0000 7f8653e00640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0c9520 0x7f865c0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.755+0000 7f8653e00640 1 -- 172.21.15.88:0/195317129 >> v1:172.21.15.132:6789/0 conn(0x7f865c15f420 legacy=0x7f865c0d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:02.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.755+0000 7f8653e00640 1 -- 172.21.15.88:0/195317129 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0d4020 con 0x7f865c0d3680 2024-04-26T20:02:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8653e00640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0d3680 0x7f865c0d3ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0c5004c884d097 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8659400640 1 -- 172.21.15.88:0/195317129 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644037070 con 0x7f865c0d3680 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8659400640 1 -- 172.21.15.88:0/195317129 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8644032440 con 0x7f865c0d3680 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8659400640 1 -- 172.21.15.88:0/195317129 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644032810 con 0x7f865c0d3680 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8660c00640 1 -- 172.21.15.88:0/195317129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0d3680 msgr2=0x7f865c0d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.756+0000 7f8660c00640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0d3680 0x7f865c0d3ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864402a180 comp rx=0 tx=0).stop 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 -- 172.21.15.88:0/195317129 shutdown_connections 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0c9520 0x7f865c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 --2- 172.21.15.88:0/195317129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0d3680 0x7f865c0d3ae0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 -- 172.21.15.88:0/195317129 >> 172.21.15.88:0/195317129 conn(0x7f865c09e3e0 msgr2=0x7f865c0d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 -- 172.21.15.88:0/195317129 shutdown_connections 2024-04-26T20:02:02.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.757+0000 7f8660c00640 1 -- 172.21.15.88:0/195317129 wait complete. 2024-04-26T20:02:02.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.758+0000 7f8660c00640 1 Processor -- start 2024-04-26T20:02:02.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.758+0000 7f8660c00640 1 -- start start 2024-04-26T20:02:02.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0d3680 0x7f865c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 0x7f865c0d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0c8d20 con 0x7f865c0c9520 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0d31a0 con 0x7f865c15f420 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.759+0000 7f8660c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0c9000 con 0x7f865c0d3680 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57126/0 (socket says 172.21.15.88:57126) 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 -- 172.21.15.88:0/929105688 learned_addr learned my addr 172.21.15.88:0/929105688 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 -- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0d3680 msgr2=0x7f865c15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0d3680 0x7f865c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f865a800640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 0x7f865c0d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 -- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 msgr2=0x7f865c0d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 0x7f865c0d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 -- 172.21.15.88:0/929105688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c160470 con 0x7f865c0c9520 2024-04-26T20:02:02.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f865a800640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 0x7f865c0d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.760+0000 7f8659e00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c430 tx=0x7f864c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.761+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c011790 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.761+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864c017070 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.761+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c011bb0 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.761+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c160750 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.761+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1f5d80 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.762+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c1f5890 con 0x7f865c0c9520 2024-04-26T20:02:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.763+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f864c01c460 con 0x7f865c0c9520 2024-04-26T20:02:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.764+0000 7f8652000640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 0x7f863806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:02.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.764+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5709+0+0 (secure 0 0 0) 0x7f864c08ba80 con 0x7f865c0c9520 2024-04-26T20:02:02.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.764+0000 7f8653e00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 0x7f863806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:02.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.765+0000 7f8653e00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 0x7f863806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:02.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.769+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f864c08de20 con 0x7f865c0c9520 2024-04-26T20:02:02.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.965+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f865c0b79f0 con 0x7f865c0c9520 2024-04-26T20:02:02.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.968+0000 7f8652000640 1 -- 172.21.15.88:0/929105688 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16666 (secure 0 0 0) 0x7f864c059ea0 con 0x7f865c0c9520 2024-04-26T20:02:02.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:02.970 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":51,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:01.583645+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:02.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 msgr2=0x7f863806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 0x7f863806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402c000 comp rx=0 tx=0).stop 2024-04-26T20:02:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 msgr2=0x7f865c15b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c430 tx=0x7f864c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 shutdown_connections 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f865c15f420 0x7f865c0d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f863806c8f0 0x7f863806edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f865c0d3680 0x7f865c15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 --2- 172.21.15.88:0/929105688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f865c0c9520 0x7f865c15b590 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.971+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 >> 172.21.15.88:0/929105688 conn(0x7f865c09e3e0 msgr2=0x7f865c158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.972+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 shutdown_connections 2024-04-26T20:02:02.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:02.972+0000 7f8660c00640 1 -- 172.21.15.88:0/929105688 wait complete. 2024-04-26T20:02:02.990 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:02:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.184+0000 7fc362800640 1 Processor -- start 2024-04-26T20:02:03.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.184+0000 7fc362800640 1 -- start start 2024-04-26T20:02:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.185+0000 7fc362800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35c0d19c0 0x7fc35c0d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.185+0000 7fc362800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc35c0d1da0 0x7fc35c162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.185+0000 7fc362800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35c0d3830 con 0x7fc35c0b6170 2024-04-26T20:02:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.185+0000 7fc362800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc35c0d40d0 con 0x7fc35c0d19c0 2024-04-26T20:02:03.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.185+0000 7fc362800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc35c0d3b10 con 0x7fc35c0d1da0 2024-04-26T20:02:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35c0d19c0 0x7fc35c0d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35c0d19c0 0x7fc35c0d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57140/0 (socket says 172.21.15.88:57140) 2024-04-26T20:02:03.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 -- 172.21.15.88:0/43492172 learned_addr learned my addr 172.21.15.88:0/43492172 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 -- 172.21.15.88:0/43492172 >> v2:172.21.15.132:3300/0 conn(0x7fc35c0d1da0 msgr2=0x7fc35c162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 --2- 172.21.15.88:0/43492172 >> v2:172.21.15.132:3300/0 conn(0x7fc35c0d1da0 0x7fc35c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 -- 172.21.15.88:0/43492172 >> v1:172.21.15.88:6789/0 conn(0x7fc35c0b6170 legacy=0x7fc35c0c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35b400640 1 -- 172.21.15.88:0/43492172 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c0d3df0 con 0x7fc35c0d19c0 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.186+0000 7fc35aa00640 1 -- 172.21.15.88:0/43492172 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758298751 0 0) 0x7fc35c0d3830 con 0x7fc35c0b6170 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.187+0000 7fc35b400640 1 --2- 172.21.15.88:0/43492172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0d19c0 0x7fc35c0d2f00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc35c08e590 tx=0x7fc34c02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b8e5dfdb15d2b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.187+0000 7fc35aa00640 1 -- 172.21.15.88:0/43492172 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c037070 con 0x7fc35c0d19c0 2024-04-26T20:02:03.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.187+0000 7fc35aa00640 1 -- 172.21.15.88:0/43492172 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c032440 con 0x7fc35c0d19c0 2024-04-26T20:02:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.187+0000 7fc35aa00640 1 -- 172.21.15.88:0/43492172 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c032810 con 0x7fc35c0d19c0 2024-04-26T20:02:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 -- 172.21.15.88:0/43492172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0d19c0 msgr2=0x7fc35c0d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:03.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 --2- 172.21.15.88:0/43492172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0d19c0 0x7fc35c0d2f00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc35c08e590 tx=0x7fc34c02a1f0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 -- 172.21.15.88:0/43492172 shutdown_connections 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 --2- 172.21.15.88:0/43492172 >> v2:172.21.15.132:3300/0 conn(0x7fc35c0d1da0 0x7fc35c162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 --2- 172.21.15.88:0/43492172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0d19c0 0x7fc35c0d2f00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 -- 172.21.15.88:0/43492172 >> 172.21.15.88:0/43492172 conn(0x7fc35c09e460 msgr2=0x7fc35c0d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 -- 172.21.15.88:0/43492172 shutdown_connections 2024-04-26T20:02:03.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.188+0000 7fc362800640 1 -- 172.21.15.88:0/43492172 wait complete. 2024-04-26T20:02:03.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.190+0000 7fc362800640 1 Processor -- start 2024-04-26T20:02:03.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.190+0000 7fc362800640 1 -- start start 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c0d19c0 0x7fc35c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c0d1da0 0x7fc35c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c162b80 con 0x7fc35c0b6170 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c0cfc30 con 0x7fc35c0d19c0 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc362800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc35c162d00 con 0x7fc35c0d1da0 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57142/0 (socket says 172.21.15.88:57142) 2024-04-26T20:02:03.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 -- 172.21.15.88:0/1861339130 learned_addr learned my addr 172.21.15.88:0/1861339130 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 -- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c0d1da0 msgr2=0x7fc35c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c0d1da0 0x7fc35c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 -- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c0d19c0 msgr2=0x7fc35c159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c0d19c0 0x7fc35c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.191+0000 7fc35b400640 1 -- 172.21.15.88:0/1861339130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c15baf0 con 0x7fc35c0b6170 2024-04-26T20:02:03.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc35b400640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c041070 con 0x7fc35c0b6170 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c03c890 con 0x7fc35c0b6170 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c037040 con 0x7fc35c0b6170 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c15a630 con 0x7fc35c0b6170 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.192+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c15c050 con 0x7fc35c0b6170 2024-04-26T20:02:03.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.193+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35c069650 con 0x7fc35c0b6170 2024-04-26T20:02:03.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.194+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc34c044070 con 0x7fc35c0b6170 2024-04-26T20:02:03.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.196+0000 7fc353400640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:03.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.196+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fc34c0ac820 con 0x7fc35c0b6170 2024-04-26T20:02:03.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.196+0000 7fc353e00640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:03.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.196+0000 7fc353e00640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 0x7fc33806ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0d5fd0 tx=0x7fc344007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:03.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.200+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc35c15baf0 con 0x7fc35c0b6170 2024-04-26T20:02:03.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:03.424+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc35c0d26d0 con 0x7fc35c0b6170 2024-04-26T20:02:04.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.156+0000 7fc353400640 1 -- 172.21.15.88:0/1861339130 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fc34c07ad60 con 0x7fc35c0b6170 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 msgr2=0x7fc33806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 0x7fc33806ede0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0d5fd0 tx=0x7fc344007930 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 msgr2=0x7fc35c155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 shutdown_connections 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35c0d1da0 0x7fc35c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35c0d19c0 0x7fc35c159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 --2- 172.21.15.88:0/1861339130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35c0b6170 0x7fc35c155200 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 >> 172.21.15.88:0/1861339130 conn(0x7fc35c09e460 msgr2=0x7fc35c0ca2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 shutdown_connections 2024-04-26T20:02:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.160+0000 7fc362800640 1 -- 172.21.15.88:0/1861339130 wait complete. 2024-04-26T20:02:04.182 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:02:04.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:02:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.386+0000 7f0070a00640 1 Processor -- start 2024-04-26T20:02:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.386+0000 7f0070a00640 1 -- start start 2024-04-26T20:02:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f0070a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f006c161140 0x7f006c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f0070a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f006c146280 0x7f006c1466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f0070a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f006c0b4d20 con 0x7f006c160d60 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f0070a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f006c146c30 con 0x7f006c161140 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f0070a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f006c0b5000 con 0x7f006c146280 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f006c160d60 0x7f006c0b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36510/0 (socket says 172.21.15.88:36510) 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006b400640 1 -- 172.21.15.88:0/1524636403 learned_addr learned my addr 172.21.15.88:0/1524636403 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006aa00640 1 --2- 172.21.15.88:0/1524636403 >> v2:172.21.15.88:3300/0 conn(0x7f006c161140 0x7f006c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006be00640 1 --2- 172.21.15.88:0/1524636403 >> v2:172.21.15.132:3300/0 conn(0x7f006c146280 0x7f006c1466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006a000640 1 -- 172.21.15.88:0/1524636403 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796773762 0 0) 0x7f006c0b4d20 con 0x7f006c160d60 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006aa00640 1 -- 172.21.15.88:0/1524636403 >> v2:172.21.15.132:3300/0 conn(0x7f006c146280 msgr2=0x7f006c1466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006aa00640 1 --2- 172.21.15.88:0/1524636403 >> v2:172.21.15.132:3300/0 conn(0x7f006c146280 0x7f006c1466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.388+0000 7f006aa00640 1 -- 172.21.15.88:0/1524636403 >> v1:172.21.15.88:6789/0 conn(0x7f006c160d60 legacy=0x7f006c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.389+0000 7f006aa00640 1 -- 172.21.15.88:0/1524636403 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c0b52e0 con 0x7f006c161140 2024-04-26T20:02:04.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.390+0000 7f006aa00640 1 --2- 172.21.15.88:0/1524636403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c06dee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f00580097a0 tx=0x7f005802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28d3eeac408e336 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:04.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.390+0000 7f006a000640 1 -- 172.21.15.88:0/1524636403 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058037070 con 0x7f006c161140 2024-04-26T20:02:04.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.390+0000 7f006a000640 1 -- 172.21.15.88:0/1524636403 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0058032440 con 0x7f006c161140 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.390+0000 7f006a000640 1 -- 172.21.15.88:0/1524636403 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058032810 con 0x7f006c161140 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 -- 172.21.15.88:0/1524636403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 msgr2=0x7f006c06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 --2- 172.21.15.88:0/1524636403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c06dee0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f00580097a0 tx=0x7f005802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 -- 172.21.15.88:0/1524636403 shutdown_connections 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 --2- 172.21.15.88:0/1524636403 >> v2:172.21.15.132:3300/0 conn(0x7f006c146280 0x7f006c1466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 --2- 172.21.15.88:0/1524636403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c06dee0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 -- 172.21.15.88:0/1524636403 >> 172.21.15.88:0/1524636403 conn(0x7f006c0b2900 msgr2=0x7f006c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 -- 172.21.15.88:0/1524636403 shutdown_connections 2024-04-26T20:02:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.391+0000 7f0070a00640 1 -- 172.21.15.88:0/1524636403 wait complete. 2024-04-26T20:02:04.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.393+0000 7f0070a00640 1 Processor -- start 2024-04-26T20:02:04.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.393+0000 7f0070a00640 1 -- start start 2024-04-26T20:02:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 0x7f006c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c160d60 0x7f006c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f006c06e420 con 0x7f006c161140 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0cf6f0 con 0x7f006c146280 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f0070a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f006c06e5a0 con 0x7f006c160d60 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.394+0000 7f006be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57172/0 (socket says 172.21.15.88:57172) 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 -- 172.21.15.88:0/2164407721 learned_addr learned my addr 172.21.15.88:0/2164407721 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006b400640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 0x7f006c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006aa00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c160d60 0x7f006c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 -- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c160d60 msgr2=0x7f006c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c160d60 0x7f006c0cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 -- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 msgr2=0x7f006c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 0x7f006c159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 -- 172.21.15.88:0/2164407721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c136550 con 0x7f006c161140 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006b400640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 0x7f006c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.395+0000 7f006be00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f006400cd50 tx=0x7f006400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.396+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064003b90 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.396+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0064003d30 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.396+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c136830 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.396+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064004240 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.396+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c136df0 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.397+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c0693c0 con 0x7f006c161140 2024-04-26T20:02:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.398+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00640043e0 con 0x7f006c161140 2024-04-26T20:02:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.399+0000 7f0063400640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 0x7f004c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.399+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f006408c590 con 0x7f006c161140 2024-04-26T20:02:04.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.399+0000 7f006b400640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 0x7f004c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:04.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.400+0000 7f006b400640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 0x7f004c06ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f005c000f60 tx=0x7f005c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:04.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.403+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f006405a9b0 con 0x7f006c161140 2024-04-26T20:02:04.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:04.645+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f006c0d18b0 con 0x7f006c161140 2024-04-26T20:02:05.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.166+0000 7f0063400640 1 -- 172.21.15.88:0/2164407721 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f006405a2b0 con 0x7f006c161140 2024-04-26T20:02:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 msgr2=0x7f004c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 0x7f004c06ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f005c000f60 tx=0x7f005c009040 comp rx=0 tx=0).stop 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 msgr2=0x7f006c0aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f006400cd50 tx=0x7f006400a660 comp rx=0 tx=0).stop 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 shutdown_connections 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c161140 0x7f006c0aab20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f004c06c920 0x7f004c06ede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f006c160d60 0x7f006c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 --2- 172.21.15.88:0/2164407721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f006c146280 0x7f006c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 >> 172.21.15.88:0/2164407721 conn(0x7f006c0b2900 msgr2=0x7f006c073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 shutdown_connections 2024-04-26T20:02:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.170+0000 7f0070a00640 1 -- 172.21.15.88:0/2164407721 wait complete. 2024-04-26T20:02:05.192 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:02:05.193 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.389+0000 7f41da400640 1 Processor -- start 2024-04-26T20:02:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.390+0000 7f41da400640 1 -- start start 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.391+0000 7f41da400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41d40d5990 0x7f41d4163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.391+0000 7f41da400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.391+0000 7f41da400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d40d3f50 con 0x7f41d415f1b0 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.391+0000 7f41da400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f41d40d36b0 con 0x7f41d40d5990 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.391+0000 7f41da400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f41d40d3990 con 0x7f41d40b82b0 2024-04-26T20:02:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.392+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.392+0000 7f41d3e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f41d40d5990 0x7f41d4163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.392+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:34158/0 (socket says 172.21.15.88:34158) 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.392+0000 7f41d9a00640 1 -- 172.21.15.88:0/1713053904 learned_addr learned my addr 172.21.15.88:0/1713053904 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.393+0000 7f41d9a00640 1 -- 172.21.15.88:0/1713053904 >> v2:172.21.15.132:3300/0 conn(0x7f41d40d5990 msgr2=0x7f41d4163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.393+0000 7f41d9a00640 1 --2- 172.21.15.88:0/1713053904 >> v2:172.21.15.132:3300/0 conn(0x7f41d40d5990 0x7f41d4163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.393+0000 7f41d9a00640 1 -- 172.21.15.88:0/1713053904 >> v1:172.21.15.132:6789/0 conn(0x7f41d415f1b0 legacy=0x7f41d4162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.393+0000 7f41d9a00640 1 -- 172.21.15.88:0/1713053904 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d40d3c70 con 0x7f41d40b82b0 2024-04-26T20:02:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.394+0000 7f41d9a00640 1 --2- 172.21.15.88:0/1713053904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009870 tx=0x7f41cc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55279f1663ddddf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.395+0000 7f41d3400640 1 -- 172.21.15.88:0/1713053904 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc037070 con 0x7f41d40b82b0 2024-04-26T20:02:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.395+0000 7f41d3400640 1 -- 172.21.15.88:0/1713053904 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc032440 con 0x7f41d40b82b0 2024-04-26T20:02:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.395+0000 7f41d3400640 1 -- 172.21.15.88:0/1713053904 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc032810 con 0x7f41d40b82b0 2024-04-26T20:02:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.395+0000 7f41da400640 1 -- 172.21.15.88:0/1713053904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 msgr2=0x7f41d40b8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.395+0000 7f41da400640 1 --2- 172.21.15.88:0/1713053904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009870 tx=0x7f41cc02a270 comp rx=0 tx=0).stop 2024-04-26T20:02:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 -- 172.21.15.88:0/1713053904 shutdown_connections 2024-04-26T20:02:05.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 --2- 172.21.15.88:0/1713053904 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40b82b0 0x7f41d40b8740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 --2- 172.21.15.88:0/1713053904 >> v2:172.21.15.132:3300/0 conn(0x7f41d40d5990 0x7f41d4163210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 -- 172.21.15.88:0/1713053904 >> 172.21.15.88:0/1713053904 conn(0x7f41d409e230 msgr2=0x7f41d40d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 -- 172.21.15.88:0/1713053904 shutdown_connections 2024-04-26T20:02:05.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.396+0000 7f41da400640 1 -- 172.21.15.88:0/1713053904 wait complete. 2024-04-26T20:02:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.398+0000 7f41da400640 1 Processor -- start 2024-04-26T20:02:05.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.398+0000 7f41da400640 1 -- start start 2024-04-26T20:02:05.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 0x7f41d40cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40d5990 0x7f41d40c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4163750 con 0x7f41d415f1b0 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f41d40d33a0 con 0x7f41d40b82b0 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41da400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f41d41638d0 con 0x7f41d40d5990 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41d9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57178/0 (socket says 172.21.15.88:57178) 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.399+0000 7f41d9a00640 1 -- 172.21.15.88:0/1579044063 learned_addr learned my addr 172.21.15.88:0/1579044063 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d3e00640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40d5990 0x7f41d40c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9000640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 0x7f41d40cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 -- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40d5990 msgr2=0x7f41d40c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40d5990 0x7f41d40c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 -- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 msgr2=0x7f41d40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 0x7f41d40cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 -- 172.21.15.88:0/1579044063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d40cb810 con 0x7f41d415f1b0 2024-04-26T20:02:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9000640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 0x7f41d40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.400+0000 7f41d9a00640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009870 tx=0x7f41cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.401+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc041020 con 0x7f41d415f1b0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.401+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41cc03d640 con 0x7f41d415f1b0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.401+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d40c8e80 con 0x7f41d415f1b0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.401+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc037040 con 0x7f41d415f1b0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.401+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d4147bc0 con 0x7f41d415f1b0 2024-04-26T20:02:05.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.402+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d40693c0 con 0x7f41d415f1b0 2024-04-26T20:02:05.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.403+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41cc044020 con 0x7f41d415f1b0 2024-04-26T20:02:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.404+0000 7f41d0c00640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.405+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f41cc0ac2f0 con 0x7f41d415f1b0 2024-04-26T20:02:05.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.405+0000 7f41d9000640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.405+0000 7f41d9000640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 0x7f41b006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001cd0 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.409+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41cc04b030 con 0x7f41d415f1b0 2024-04-26T20:02:05.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.649+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41d408d090 con 0x7f41d415f1b0 2024-04-26T20:02:05.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.650+0000 7f41d0c00640 1 -- 172.21.15.88:0/1579044063 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5181 (secure 0 0 0) 0x7f41cc099020 con 0x7f41d415f1b0 2024-04-26T20:02:05.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:05.651 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:05.652 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 29 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 msgr2=0x7f41b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 0x7f41b006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f41c4001cd0 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 msgr2=0x7f41d41f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009870 tx=0x7f41cc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 shutdown_connections 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d415f1b0 0x7f41d41f49f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f41b006c8f0 0x7f41b006edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f41d40d5990 0x7f41d40c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 --2- 172.21.15.88:0/1579044063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40b82b0 0x7f41d40cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 >> 172.21.15.88:0/1579044063 conn(0x7f41d409e230 msgr2=0x7f41d40b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:05.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 shutdown_connections 2024-04-26T20:02:05.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.653+0000 7f41da400640 1 -- 172.21.15.88:0/1579044063 wait complete. 2024-04-26T20:02:05.673 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:05.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.864+0000 7f0b57400640 1 Processor -- start 2024-04-26T20:02:05.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.864+0000 7f0b57400640 1 -- start start 2024-04-26T20:02:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b57400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0b5015cfd0 0x7f0b5006f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b57400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b500ae850 con 0x7f0b5006c770 2024-04-26T20:02:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b57400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b500aeb30 con 0x7f0b5015cb70 2024-04-26T20:02:05.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b57400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f0b500af460 con 0x7f0b5015cfd0 2024-04-26T20:02:05.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b54a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0b5006c770 0x7f0b5006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36516/0 (socket says 172.21.15.88:36516) 2024-04-26T20:02:05.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b54a00640 1 -- 172.21.15.88:0/2794213430 learned_addr learned my addr 172.21.15.88:0/2794213430 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b55400640 1 --2- 172.21.15.88:0/2794213430 >> v2:172.21.15.132:3300/0 conn(0x7f0b5015cfd0 0x7f0b5006f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504741681 0 0) 0x7f0b500ae850 con 0x7f0b5006c770 2024-04-26T20:02:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.866+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b38003520 con 0x7f0b5006c770 2024-04-26T20:02:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3709620121 0 0) 0x7f0b38003520 con 0x7f0b5006c770 2024-04-26T20:02:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 >> v2:172.21.15.132:3300/0 conn(0x7f0b5015cfd0 msgr2=0x7f0b5006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 --2- 172.21.15.88:0/2794213430 >> v2:172.21.15.132:3300/0 conn(0x7f0b5015cfd0 0x7f0b5006f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 >> v1:172.21.15.132:6789/0 conn(0x7f0b5015cb70 legacy=0x7f0b5006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b500aee10 con 0x7f0b5006c770 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f0b44001670 con 0x7f0b5006c770 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 >> v1:172.21.15.88:6789/0 conn(0x7f0b5006c770 legacy=0x7f0b5006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b380039b0 0x7f0b38003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b500af460 con 0x7f0b38005210 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3800c140 con 0x7f0b380039b0 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3800c340 con 0x7f0b38004800 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.867+0000 7f0b55400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b54a00640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b380039b0 0x7f0b38003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b4fe00640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b55400640 1 -- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b38004800 msgr2=0x7f0b38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b55400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b55400640 1 -- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b380039b0 msgr2=0x7f0b38003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b55400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b380039b0 0x7f0b38003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.868+0000 7f0b55400640 1 -- 172.21.15.88:0/2794213430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38003520 con 0x7f0b38005210 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.869+0000 7f0b55400640 1 -- 172.21.15.88:0/2794213430 reap_dead start 2024-04-26T20:02:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.869+0000 7f0b55400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b3800b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007360 tx=0x7f0b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c1164f5a3c8d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.869+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48009be0 con 0x7f0b38005210 2024-04-26T20:02:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.869+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b48003b90 con 0x7f0b38005210 2024-04-26T20:02:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.869+0000 7f0b4f400640 1 -- 172.21.15.88:0/2794213430 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004020 con 0x7f0b38005210 2024-04-26T20:02:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 -- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 msgr2=0x7f0b3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b3800b9c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0b48007360 tx=0x7f0b38005830 comp rx=0 tx=0).stop 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 -- 172.21.15.88:0/2794213430 shutdown_connections 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 --2- 172.21.15.88:0/2794213430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b3800b9c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 -- 172.21.15.88:0/2794213430 >> 172.21.15.88:0/2794213430 conn(0x7f0b500b2d50 msgr2=0x7f0b500b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 -- 172.21.15.88:0/2794213430 shutdown_connections 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.870+0000 7f0b57400640 1 -- 172.21.15.88:0/2794213430 wait complete. 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.872+0000 7f0b57400640 1 Processor -- start 2024-04-26T20:02:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.872+0000 7f0b57400640 1 -- start start 2024-04-26T20:02:05.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 0x7f0b50072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b500731b0 0x7f0b50073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5015c780 con 0x7f0b38005210 2024-04-26T20:02:05.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5015c600 con 0x7f0b500727f0 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b57400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0b500af8f0 con 0x7f0b500731b0 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57198/0 (socket says 172.21.15.88:57198) 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 -- 172.21.15.88:0/1171727814 learned_addr learned my addr 172.21.15.88:0/1171727814 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b4fe00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 0x7f0b50072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 -- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b500731b0 msgr2=0x7f0b50073630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b500731b0 0x7f0b50073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 -- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 msgr2=0x7f0b50072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 0x7f0b50072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b54a00640 1 -- 172.21.15.88:0/1171727814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b501e6960 con 0x7f0b38005210 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.873+0000 7f0b4fe00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 0x7f0b50072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b54a00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400e7f0 tx=0x7f0b4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400c8d0 con 0x7f0b38005210 2024-04-26T20:02:05.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b44018070 con 0x7f0b38005210 2024-04-26T20:02:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b501e6c40 con 0x7f0b38005210 2024-04-26T20:02:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400ccb0 con 0x7f0b38005210 2024-04-26T20:02:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.874+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b500722d0 con 0x7f0b38005210 2024-04-26T20:02:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.875+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b50069540 con 0x7f0b38005210 2024-04-26T20:02:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.876+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b4401d460 con 0x7f0b38005210 2024-04-26T20:02:05.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.877+0000 7f0b4cc00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 0x7f0b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:05.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.877+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f0b4408b9e0 con 0x7f0b38005210 2024-04-26T20:02:05.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.877+0000 7f0b4fe00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 0x7f0b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:05.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.878+0000 7f0b4fe00640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 0x7f0b3006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:05.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:05.881+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b44059e00 con 0x7f0b38005210 2024-04-26T20:02:06.064 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-04-26T20:02:06.063+0000 7f2acb200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:02:06.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.084+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b500cd430 con 0x7f0b38005210 2024-04-26T20:02:06.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.086+0000 7f0b4cc00640 1 -- 172.21.15.88:0/1171727814 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7f0b44059700 con 0x7f0b38005210 2024-04-26T20:02:06.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:06.087 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:06.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 msgr2=0x7f0b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 0x7f0b3006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c000c00 tx=0x7f0b3c009040 comp rx=0 tx=0).stop 2024-04-26T20:02:06.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 msgr2=0x7f0b501e61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400e7f0 tx=0x7f0b4400ecb0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 shutdown_connections 2024-04-26T20:02:06.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0b3006c8f0 0x7f0b3006edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0b500731b0 0x7f0b50073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b500727f0 0x7f0b50072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 --2- 172.21.15.88:0/1171727814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b38005210 0x7f0b501e61f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 >> 172.21.15.88:0/1171727814 conn(0x7f0b500b2d50 msgr2=0x7f0b5015f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.089+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 shutdown_connections 2024-04-26T20:02:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.090+0000 7f0b57400640 1 -- 172.21.15.88:0/1171727814 wait complete. 2024-04-26T20:02:06.109 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:02:06.109 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.303+0000 7fa922400640 1 Processor -- start 2024-04-26T20:02:06.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.303+0000 7fa922400640 1 -- start start 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.304+0000 7fa922400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa91c146680 0x7fa91c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa922400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c0d0e70 0x7fa91c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa922400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91c146a60 con 0x7fa91c160930 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa922400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa91c06e970 con 0x7fa91c146680 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa922400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c0d17f0 con 0x7fa91c0d0e70 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa91c146680 0x7fa91c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa91c160930 0x7fa91c160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36522/0 (socket says 172.21.15.88:36522) 2024-04-26T20:02:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921a00640 1 -- 172.21.15.88:0/2572317793 learned_addr learned my addr 172.21.15.88:0/2572317793 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921000640 1 -- 172.21.15.88:0/2572317793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c0d0e70 msgr2=0x7fa91c0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921000640 1 --2- 172.21.15.88:0/2572317793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c0d0e70 0x7fa91c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921000640 1 -- 172.21.15.88:0/2572317793 >> v1:172.21.15.88:6789/0 conn(0x7fa91c160930 legacy=0x7fa91c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.305+0000 7fa921000640 1 -- 172.21.15.88:0/2572317793 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c06dd60 con 0x7fa91c146680 2024-04-26T20:02:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.306+0000 7fa921000640 1 --2- 172.21.15.88:0/2572317793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c0b5990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8d2421c44a37cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa91b400640 1 -- 172.21.15.88:0/2572317793 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910037070 con 0x7fa91c146680 2024-04-26T20:02:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa91b400640 1 -- 172.21.15.88:0/2572317793 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910032440 con 0x7fa91c146680 2024-04-26T20:02:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa91b400640 1 -- 172.21.15.88:0/2572317793 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910032810 con 0x7fa91c146680 2024-04-26T20:02:06.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa922400640 1 -- 172.21.15.88:0/2572317793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 msgr2=0x7fa91c0b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa922400640 1 --2- 172.21.15.88:0/2572317793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c0b5990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002a180 comp rx=0 tx=0).stop 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa922400640 1 -- 172.21.15.88:0/2572317793 shutdown_connections 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa922400640 1 --2- 172.21.15.88:0/2572317793 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c0d0e70 0x7fa91c0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.307+0000 7fa922400640 1 --2- 172.21.15.88:0/2572317793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c0b5990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.308+0000 7fa922400640 1 -- 172.21.15.88:0/2572317793 >> 172.21.15.88:0/2572317793 conn(0x7fa91c06c740 msgr2=0x7fa91c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.308+0000 7fa922400640 1 -- 172.21.15.88:0/2572317793 shutdown_connections 2024-04-26T20:02:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.308+0000 7fa922400640 1 -- 172.21.15.88:0/2572317793 wait complete. 2024-04-26T20:02:06.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.309+0000 7fa922400640 1 Processor -- start 2024-04-26T20:02:06.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.309+0000 7fa922400640 1 -- start start 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa91c0d0e70 0x7fa91c145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c160930 0x7fa91c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c06e600 con 0x7fa91c146680 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c06e320 con 0x7fa91c0d0e70 2024-04-26T20:02:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.310+0000 7fa922400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa91c0cd6a0 con 0x7fa91c160930 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57216/0 (socket says 172.21.15.88:57216) 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 -- 172.21.15.88:0/1075815908 learned_addr learned my addr 172.21.15.88:0/1075815908 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 -- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c160930 msgr2=0x7fa91c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c160930 0x7fa91c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 -- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa91c0d0e70 msgr2=0x7fa91c145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa91c0d0e70 0x7fa91c145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 -- 172.21.15.88:0/1075815908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1456c0 con 0x7fa91c146680 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.311+0000 7fa91be00640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b4a0 tx=0x7fa90800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.312+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908014070 con 0x7fa91c146680 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.312+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908010610 con 0x7fa91c146680 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.312+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908010a00 con 0x7fa91c146680 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.312+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c0cb810 con 0x7fa91c146680 2024-04-26T20:02:06.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.312+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c0cad30 con 0x7fa91c146680 2024-04-26T20:02:06.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.313+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c0693c0 con 0x7fa91c146680 2024-04-26T20:02:06.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.314+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa908017070 con 0x7fa91c146680 2024-04-26T20:02:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.315+0000 7fa918c00640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 0x7fa8fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.315+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fa90808b5b0 con 0x7fa91c146680 2024-04-26T20:02:06.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.315+0000 7fa921000640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 0x7fa8fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.316+0000 7fa921000640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 0x7fa8fc06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa910034000 tx=0x7fa91002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.319+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9080599d0 con 0x7fa91c146680 2024-04-26T20:02:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.509+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa91c15ea30 con 0x7fa8fc06c8f0 2024-04-26T20:02:06.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.518+0000 7fa918c00640 1 -- 172.21.15.88:0/1075815908 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7fa91c15ea30 con 0x7fa8fc06c8f0 2024-04-26T20:02:06.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:06.523 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.522+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 msgr2=0x7fa8fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.522+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 0x7fa8fc06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa910034000 tx=0x7fa91002c000 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.522+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 msgr2=0x7fa91c1f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.522+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b4a0 tx=0x7fa90800b960 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 shutdown_connections 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa8fc06c8f0 0x7fa8fc06edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa91c160930 0x7fa91c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa91c146680 0x7fa91c1f84f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 --2- 172.21.15.88:0/1075815908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa91c0d0e70 0x7fa91c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 >> 172.21.15.88:0/1075815908 conn(0x7fa91c06c740 msgr2=0x7fa91c0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 shutdown_connections 2024-04-26T20:02:06.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.523+0000 7fa922400640 1 -- 172.21.15.88:0/1075815908 wait complete. 2024-04-26T20:02:06.542 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-04-26T20:02:04.980407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225980,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748740,"statfs":{"total":773094113280,"available":772862709760,"internally_reserved":0,"allocated":5775360,"data_stored":3098824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.883228"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T20:01:57.336153+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:57.336153+0000","last_peered":"2024-04-26T20:01:57.336153+0000","last_clean":"2024-04-26T20:01:57.336153+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:57.336153+0000","last_undegraded":"2024-04-26T20:01:57.336153+0000","last_fullsized":"2024-04-26T20:01:57.336153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":830129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1167360,"data_stored":830129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:06.546 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:06.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.778+0000 7ff385000640 1 Processor -- start 2024-04-26T20:02:06.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.778+0000 7ff385000640 1 -- start start 2024-04-26T20:02:06.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff385000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff3800d0e70 0x7ff3800d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff385000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3800af6e0 con 0x7ff3800b5d40 2024-04-26T20:02:06.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff385000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3800d1790 con 0x7ff3800b5660 2024-04-26T20:02:06.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff385000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff3800727f0 con 0x7ff3800d0e70 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff3800d0e70 0x7ff3800d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff3800b5d40 0x7ff3800aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:36538/0 (socket says 172.21.15.88:36538) 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37fe00640 1 -- 172.21.15.88:0/768452452 learned_addr learned my addr 172.21.15.88:0/768452452 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37f400640 1 -- 172.21.15.88:0/768452452 >> v1:172.21.15.132:6789/0 conn(0x7ff3800b5660 legacy=0x7ff3800ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37f400640 1 -- 172.21.15.88:0/768452452 >> v1:172.21.15.88:6789/0 conn(0x7ff3800b5d40 legacy=0x7ff3800aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37f400640 1 -- 172.21.15.88:0/768452452 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3800b4a10 con 0x7ff3800d0e70 2024-04-26T20:02:06.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.780+0000 7ff37e000640 1 -- 172.21.15.88:0/768452452 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724968904 0 0) 0x7ff3800af6e0 con 0x7ff3800b5d40 2024-04-26T20:02:06.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.781+0000 7ff37f400640 1 --2- 172.21.15.88:0/768452452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800d0e70 0x7ff3800d1250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3700097a0 tx=0x7ff37002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b7f63a6830dd42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.782+0000 7ff37e000640 1 -- 172.21.15.88:0/768452452 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370037070 con 0x7ff3800d0e70 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.782+0000 7ff37e000640 1 -- 172.21.15.88:0/768452452 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff37002ae50 con 0x7ff3800d0e70 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.782+0000 7ff37e000640 1 -- 172.21.15.88:0/768452452 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370032660 con 0x7ff3800d0e70 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.782+0000 7ff385000640 1 -- 172.21.15.88:0/768452452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800d0e70 msgr2=0x7ff3800d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.782+0000 7ff385000640 1 --2- 172.21.15.88:0/768452452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800d0e70 0x7ff3800d1250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff3700097a0 tx=0x7ff37002a140 comp rx=0 tx=0).stop 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.783+0000 7ff385000640 1 -- 172.21.15.88:0/768452452 shutdown_connections 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.783+0000 7ff385000640 1 --2- 172.21.15.88:0/768452452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800d0e70 0x7ff3800d1250 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.783+0000 7ff385000640 1 -- 172.21.15.88:0/768452452 >> 172.21.15.88:0/768452452 conn(0x7ff38006c740 msgr2=0x7ff38006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.783+0000 7ff385000640 1 -- 172.21.15.88:0/768452452 shutdown_connections 2024-04-26T20:02:06.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.783+0000 7ff385000640 1 -- 172.21.15.88:0/768452452 wait complete. 2024-04-26T20:02:06.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.785+0000 7ff385000640 1 Processor -- start 2024-04-26T20:02:06.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.785+0000 7ff385000640 1 -- start start 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3800b5660 0x7ff380149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3800d0e70 0x7ff3801f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3800b0430 con 0x7ff3800b5d40 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3800b4fd0 con 0x7ff3800d0e70 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff385000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff3800b05b0 con 0x7ff3800b5660 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57230/0 (socket says 172.21.15.88:57230) 2024-04-26T20:02:06.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.786+0000 7ff37ea00640 1 -- 172.21.15.88:0/3784197157 learned_addr learned my addr 172.21.15.88:0/3784197157 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 -- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3800b5660 msgr2=0x7ff380149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3800b5660 0x7ff380149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 -- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3800d0e70 msgr2=0x7ff3801f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3800d0e70 0x7ff3801f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 -- 172.21.15.88:0/3784197157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380160670 con 0x7ff3800b5d40 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff37ea00640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c460 tx=0x7ff36c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c016020 con 0x7ff3800b5d40 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c012870 con 0x7ff3800b5d40 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c012c60 con 0x7ff3800b5d40 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff380160950 con 0x7ff3800b5d40 2024-04-26T20:02:06.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.787+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff380158030 con 0x7ff3800b5d40 2024-04-26T20:02:06.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.788+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff380158230 con 0x7ff3800b5d40 2024-04-26T20:02:06.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.789+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff36c019020 con 0x7ff3800b5d40 2024-04-26T20:02:06.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.790+0000 7ff377400640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 0x7ff36006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:06.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.790+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7ff36c08b8a0 con 0x7ff3800b5d40 2024-04-26T20:02:06.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.791+0000 7ff37f400640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 0x7ff36006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:06.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.791+0000 7ff37f400640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 0x7ff36006edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff370034000 tx=0x7ff37002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:06.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.795+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff380158230 con 0x7ff3800b5d40 2024-04-26T20:02:06.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.977+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3801499b0 con 0x7ff36006c8f0 2024-04-26T20:02:06.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:06.995+0000 7ff377400640 1 -- 172.21.15.88:0/3784197157 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7ff3801499b0 con 0x7ff36006c8f0 2024-04-26T20:02:06.999 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:07.001 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:07.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 msgr2=0x7ff36006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 0x7ff36006edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff370034000 tx=0x7ff37002c000 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 msgr2=0x7ff3801f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c460 tx=0x7ff36c00c920 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 shutdown_connections 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3800d0e70 0x7ff3801f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3800b5d40 0x7ff3801f8440 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36006c8f0 0x7ff36006edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 --2- 172.21.15.88:0/3784197157 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3800b5660 0x7ff380149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 >> 172.21.15.88:0/3784197157 conn(0x7ff38006c740 msgr2=0x7ff38015e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 shutdown_connections 2024-04-26T20:02:07.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.002+0000 7ff385000640 1 -- 172.21.15.88:0/3784197157 wait complete. 2024-04-26T20:02:07.022 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-04-26T20:02:04.980407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225980,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748740,"statfs":{"total":773094113280,"available":772862709760,"internally_reserved":0,"allocated":5775360,"data_stored":3098824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.883228"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:01.097037+0000","last_change":"2024-04-26T20:02:01.097037+0000","last_active":"2024-04-26T20:02:01.097037+0000","last_peered":"2024-04-26T20:02:01.097037+0000","last_clean":"2024-04-26T20:02:01.097037+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:01.097037+0000","last_undegraded":"2024-04-26T20:02:01.097037+0000","last_fullsized":"2024-04-26T20:02:01.097037+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:02:00.081196+0000","last_change":"2024-04-26T20:02:00.081196+0000","last_active":"2024-04-26T20:02:00.081196+0000","last_peered":"2024-04-26T20:02:00.081196+0000","last_clean":"2024-04-26T20:02:00.081196+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:02:00.081196+0000","last_undegraded":"2024-04-26T20:02:00.081196+0000","last_fullsized":"2024-04-26T20:02:00.081196+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T20:01:57.336153+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:01:57.336153+0000","last_peered":"2024-04-26T20:01:57.336153+0000","last_clean":"2024-04-26T20:01:57.336153+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:01:57.336153+0000","last_undegraded":"2024-04-26T20:01:57.336153+0000","last_fullsized":"2024-04-26T20:01:57.336153+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":830129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":239761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1167360,"data_stored":830129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:07.026 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:07.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.267+0000 7fd66fe00640 1 Processor -- start 2024-04-26T20:02:07.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.267+0000 7fd66fe00640 1 -- start start 2024-04-26T20:02:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.268+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6700b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.268+0000 7fd66fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6700d0e70 0x7fd6700d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.268+0000 7fd66fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd670146a60 con 0x7fd670160930 2024-04-26T20:02:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.268+0000 7fd66fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd67006e970 con 0x7fd6700d0e70 2024-04-26T20:02:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.268+0000 7fd66fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd6700d17f0 con 0x7fd670146680 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.269+0000 7fd66ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6700b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.269+0000 7fd66e000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd6700d0e70 0x7fd6700d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.269+0000 7fd66ea00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6700b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:34204/0 (socket says 172.21.15.88:34204) 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.269+0000 7fd66ea00640 1 -- 172.21.15.88:0/3398318066 learned_addr learned my addr 172.21.15.88:0/3398318066 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.270+0000 7fd66e000640 1 -- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 msgr2=0x7fd6700b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.270+0000 7fd66e000640 1 --2- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6700b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.270+0000 7fd66e000640 1 -- 172.21.15.88:0/3398318066 >> v1:172.21.15.132:6789/0 conn(0x7fd670160930 legacy=0x7fd670160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.270+0000 7fd66e000640 1 -- 172.21.15.88:0/3398318066 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67006dd60 con 0x7fd6700d0e70 2024-04-26T20:02:07.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.271+0000 7fd66e000640 1 --2- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6700d0e70 0x7fd6700d12b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097a0 tx=0x7fd65802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6947cbd42c101b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.272+0000 7fd66d600640 1 -- 172.21.15.88:0/3398318066 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658037070 con 0x7fd6700d0e70 2024-04-26T20:02:07.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.272+0000 7fd66d600640 1 -- 172.21.15.88:0/3398318066 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd658032440 con 0x7fd6700d0e70 2024-04-26T20:02:07.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.272+0000 7fd66d600640 1 -- 172.21.15.88:0/3398318066 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658032810 con 0x7fd6700d0e70 2024-04-26T20:02:07.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.272+0000 7fd66fe00640 1 -- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6700d0e70 msgr2=0x7fd6700d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.272+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6700d0e70 0x7fd6700d12b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd6580097a0 tx=0x7fd65802a180 comp rx=0 tx=0).stop 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 -- 172.21.15.88:0/3398318066 shutdown_connections 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6700d0e70 0x7fd6700d12b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3398318066 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6700b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 -- 172.21.15.88:0/3398318066 >> 172.21.15.88:0/3398318066 conn(0x7fd67006c740 msgr2=0x7fd67006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 -- 172.21.15.88:0/3398318066 shutdown_connections 2024-04-26T20:02:07.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.273+0000 7fd66fe00640 1 -- 172.21.15.88:0/3398318066 wait complete. 2024-04-26T20:02:07.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.274+0000 7fd66fe00640 1 Processor -- start 2024-04-26T20:02:07.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.274+0000 7fd66fe00640 1 -- start start 2024-04-26T20:02:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6701587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd670160930 0x7fd670164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd67006e600 con 0x7fd6700d0e70 2024-04-26T20:02:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd67006e320 con 0x7fd670160930 2024-04-26T20:02:07.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.275+0000 7fd66fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd6700cd6a0 con 0x7fd670146680 2024-04-26T20:02:07.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57244/0 (socket says 172.21.15.88:57244) 2024-04-26T20:02:07.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 -- 172.21.15.88:0/3490067529 learned_addr learned my addr 172.21.15.88:0/3490067529 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 -- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 msgr2=0x7fd6701587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6701587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 -- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd670160930 msgr2=0x7fd670164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd670160930 0x7fd670164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 -- 172.21.15.88:0/3490067529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd670162850 con 0x7fd6700d0e70 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.276+0000 7fd66ea00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd660009d90 tx=0x7fd66000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.277+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000cda0 con 0x7fd6700d0e70 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.277+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660019070 con 0x7fd6700d0e70 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.277+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd670162b30 con 0x7fd6700d0e70 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.277+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660014680 con 0x7fd6700d0e70 2024-04-26T20:02:07.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.277+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701630f0 con 0x7fd6700d0e70 2024-04-26T20:02:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.278+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6700693c0 con 0x7fd6700d0e70 2024-04-26T20:02:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.279+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd660005d10 con 0x7fd6700d0e70 2024-04-26T20:02:07.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.280+0000 7fd667400640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 0x7fd64c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.280+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fd66008c0b0 con 0x7fd6700d0e70 2024-04-26T20:02:07.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.280+0000 7fd66e000640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 0x7fd64c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.281+0000 7fd66e000640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 0x7fd64c06ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd658034000 tx=0x7fd65802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.285+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd660010040 con 0x7fd6700d0e70 2024-04-26T20:02:07.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.481+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6700d1570 con 0x7fd64c06c920 2024-04-26T20:02:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.501+0000 7fd667400640 1 -- 172.21.15.88:0/3490067529 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7fd6700d1570 con 0x7fd64c06c920 2024-04-26T20:02:07.503 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:07.506 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.507+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 msgr2=0x7fd64c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.507+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 0x7fd64c06ede0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd658034000 tx=0x7fd65802c000 comp rx=0 tx=0).stop 2024-04-26T20:02:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.507+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 msgr2=0x7fd67015bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.507+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd660009d90 tx=0x7fd66000c490 comp rx=0 tx=0).stop 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 shutdown_connections 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd670160930 0x7fd670164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd64c06c920 0x7fd64c06ede0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd670146680 0x7fd6701587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 --2- 172.21.15.88:0/3490067529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6700d0e70 0x7fd67015bc90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 >> 172.21.15.88:0/3490067529 conn(0x7fd67006c740 msgr2=0x7fd670160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 shutdown_connections 2024-04-26T20:02:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.508+0000 7fd66fe00640 1 -- 172.21.15.88:0/3490067529 wait complete. 2024-04-26T20:02:07.528 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-26T20:02:06.993956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226424,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748296,"statfs":{"total":773094113280,"available":772862255104,"internally_reserved":0,"allocated":6234112,"data_stored":3304916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.014704"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236291+0000","last_change":"2024-04-26T20:02:02.143484+0000","last_active":"2024-04-26T20:02:03.236291+0000","last_peered":"2024-04-26T20:02:03.236291+0000","last_clean":"2024-04-26T20:02:03.236291+0000","last_became_active":"2024-04-26T20:02:02.143218+0000","last_became_peered":"2024-04-26T20:02:02.143218+0000","last_unstale":"2024-04-26T20:02:03.236291+0000","last_undegraded":"2024-04-26T20:02:03.236291+0000","last_fullsized":"2024-04-26T20:02:03.236291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:03.607368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238903+0000","last_change":"2024-04-26T20:02:02.137481+0000","last_active":"2024-04-26T20:02:03.238903+0000","last_peered":"2024-04-26T20:02:03.238903+0000","last_clean":"2024-04-26T20:02:03.238903+0000","last_became_active":"2024-04-26T20:02:02.136583+0000","last_became_peered":"2024-04-26T20:02:02.136583+0000","last_unstale":"2024-04-26T20:02:03.238903+0000","last_undegraded":"2024-04-26T20:02:03.238903+0000","last_fullsized":"2024-04-26T20:02:03.238903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:48.053920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239432+0000","last_change":"2024-04-26T20:02:02.168154+0000","last_active":"2024-04-26T20:02:03.239432+0000","last_peered":"2024-04-26T20:02:03.239432+0000","last_clean":"2024-04-26T20:02:03.239432+0000","last_became_active":"2024-04-26T20:02:02.167855+0000","last_became_peered":"2024-04-26T20:02:02.167855+0000","last_unstale":"2024-04-26T20:02:03.239432+0000","last_undegraded":"2024-04-26T20:02:03.239432+0000","last_fullsized":"2024-04-26T20:02:03.239432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237419+0000","last_change":"2024-04-26T20:02:02.157856+0000","last_active":"2024-04-26T20:02:03.237419+0000","last_peered":"2024-04-26T20:02:03.237419+0000","last_clean":"2024-04-26T20:02:03.237419+0000","last_became_active":"2024-04-26T20:02:02.156779+0000","last_became_peered":"2024-04-26T20:02:02.156779+0000","last_unstale":"2024-04-26T20:02:03.237419+0000","last_undegraded":"2024-04-26T20:02:03.237419+0000","last_fullsized":"2024-04-26T20:02:03.237419+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:26.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236036+0000","last_change":"2024-04-26T20:02:02.164743+0000","last_active":"2024-04-26T20:02:03.236036+0000","last_peered":"2024-04-26T20:02:03.236036+0000","last_clean":"2024-04-26T20:02:03.236036+0000","last_became_active":"2024-04-26T20:02:02.164403+0000","last_became_peered":"2024-04-26T20:02:02.164403+0000","last_unstale":"2024-04-26T20:02:03.236036+0000","last_undegraded":"2024-04-26T20:02:03.236036+0000","last_fullsized":"2024-04-26T20:02:03.236036+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:10.126360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238106+0000","last_change":"2024-04-26T20:02:02.153689+0000","last_active":"2024-04-26T20:02:03.238106+0000","last_peered":"2024-04-26T20:02:03.238106+0000","last_clean":"2024-04-26T20:02:03.238106+0000","last_became_active":"2024-04-26T20:02:02.153476+0000","last_became_peered":"2024-04-26T20:02:02.153476+0000","last_unstale":"2024-04-26T20:02:03.238106+0000","last_undegraded":"2024-04-26T20:02:03.238106+0000","last_fullsized":"2024-04-26T20:02:03.238106+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151473+0000","last_change":"2024-04-26T20:02:02.163749+0000","last_active":"2024-04-26T20:02:03.151473+0000","last_peered":"2024-04-26T20:02:03.151473+0000","last_clean":"2024-04-26T20:02:03.151473+0000","last_became_active":"2024-04-26T20:02:02.162498+0000","last_became_peered":"2024-04-26T20:02:02.162498+0000","last_unstale":"2024-04-26T20:02:03.151473+0000","last_undegraded":"2024-04-26T20:02:03.151473+0000","last_fullsized":"2024-04-26T20:02:03.151473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235764+0000","last_change":"2024-04-26T20:02:02.155433+0000","last_active":"2024-04-26T20:02:03.235764+0000","last_peered":"2024-04-26T20:02:03.235764+0000","last_clean":"2024-04-26T20:02:03.235764+0000","last_became_active":"2024-04-26T20:02:02.155167+0000","last_became_peered":"2024-04-26T20:02:02.155167+0000","last_unstale":"2024-04-26T20:02:03.235764+0000","last_undegraded":"2024-04-26T20:02:03.235764+0000","last_fullsized":"2024-04-26T20:02:03.235764+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149596+0000","last_change":"2024-04-26T20:02:02.145050+0000","last_active":"2024-04-26T20:02:03.149596+0000","last_peered":"2024-04-26T20:02:03.149596+0000","last_clean":"2024-04-26T20:02:03.149596+0000","last_became_active":"2024-04-26T20:02:02.144748+0000","last_became_peered":"2024-04-26T20:02:02.144748+0000","last_unstale":"2024-04-26T20:02:03.149596+0000","last_undegraded":"2024-04-26T20:02:03.149596+0000","last_fullsized":"2024-04-26T20:02:03.149596+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:27.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147062+0000","last_change":"2024-04-26T20:02:02.157240+0000","last_active":"2024-04-26T20:02:03.147062+0000","last_peered":"2024-04-26T20:02:03.147062+0000","last_clean":"2024-04-26T20:02:03.147062+0000","last_became_active":"2024-04-26T20:02:02.156963+0000","last_became_peered":"2024-04-26T20:02:02.156963+0000","last_unstale":"2024-04-26T20:02:03.147062+0000","last_undegraded":"2024-04-26T20:02:03.147062+0000","last_fullsized":"2024-04-26T20:02:03.147062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:01.598249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147030+0000","last_change":"2024-04-26T20:02:02.149796+0000","last_active":"2024-04-26T20:02:03.147030+0000","last_peered":"2024-04-26T20:02:03.147030+0000","last_clean":"2024-04-26T20:02:03.147030+0000","last_became_active":"2024-04-26T20:02:02.146559+0000","last_became_peered":"2024-04-26T20:02:02.146559+0000","last_unstale":"2024-04-26T20:02:03.147030+0000","last_undegraded":"2024-04-26T20:02:03.147030+0000","last_fullsized":"2024-04-26T20:02:03.147030+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:23.159614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237551+0000","last_change":"2024-04-26T20:02:02.163072+0000","last_active":"2024-04-26T20:02:03.237551+0000","last_peered":"2024-04-26T20:02:03.237551+0000","last_clean":"2024-04-26T20:02:03.237551+0000","last_became_active":"2024-04-26T20:02:02.162527+0000","last_became_peered":"2024-04-26T20:02:02.162527+0000","last_unstale":"2024-04-26T20:02:03.237551+0000","last_undegraded":"2024-04-26T20:02:03.237551+0000","last_fullsized":"2024-04-26T20:02:03.237551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:39.029864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151035+0000","last_change":"2024-04-26T20:02:02.165794+0000","last_active":"2024-04-26T20:02:03.151035+0000","last_peered":"2024-04-26T20:02:03.151035+0000","last_clean":"2024-04-26T20:02:03.151035+0000","last_became_active":"2024-04-26T20:02:02.165488+0000","last_became_peered":"2024-04-26T20:02:02.165488+0000","last_unstale":"2024-04-26T20:02:03.151035+0000","last_undegraded":"2024-04-26T20:02:03.151035+0000","last_fullsized":"2024-04-26T20:02:03.151035+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.541941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149361+0000","last_change":"2024-04-26T20:02:02.144523+0000","last_active":"2024-04-26T20:02:03.149361+0000","last_peered":"2024-04-26T20:02:03.149361+0000","last_clean":"2024-04-26T20:02:03.149361+0000","last_became_active":"2024-04-26T20:02:02.143456+0000","last_became_peered":"2024-04-26T20:02:02.143456+0000","last_unstale":"2024-04-26T20:02:03.149361+0000","last_undegraded":"2024-04-26T20:02:03.149361+0000","last_fullsized":"2024-04-26T20:02:03.149361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:41.021198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.152033+0000","last_change":"2024-04-26T20:02:02.141347+0000","last_active":"2024-04-26T20:02:03.152033+0000","last_peered":"2024-04-26T20:02:03.152033+0000","last_clean":"2024-04-26T20:02:03.152033+0000","last_became_active":"2024-04-26T20:02:02.141029+0000","last_became_peered":"2024-04-26T20:02:02.141029+0000","last_unstale":"2024-04-26T20:02:03.152033+0000","last_undegraded":"2024-04-26T20:02:03.152033+0000","last_fullsized":"2024-04-26T20:02:03.152033+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238361+0000","last_change":"2024-04-26T20:02:02.156328+0000","last_active":"2024-04-26T20:02:03.238361+0000","last_peered":"2024-04-26T20:02:03.238361+0000","last_clean":"2024-04-26T20:02:03.238361+0000","last_became_active":"2024-04-26T20:02:02.155610+0000","last_became_peered":"2024-04-26T20:02:02.155610+0000","last_unstale":"2024-04-26T20:02:03.238361+0000","last_undegraded":"2024-04-26T20:02:03.238361+0000","last_fullsized":"2024-04-26T20:02:03.238361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239089+0000","last_change":"2024-04-26T20:02:02.163223+0000","last_active":"2024-04-26T20:02:03.239089+0000","last_peered":"2024-04-26T20:02:03.239089+0000","last_clean":"2024-04-26T20:02:03.239089+0000","last_became_active":"2024-04-26T20:02:02.160865+0000","last_became_peered":"2024-04-26T20:02:02.160865+0000","last_unstale":"2024-04-26T20:02:03.239089+0000","last_undegraded":"2024-04-26T20:02:03.239089+0000","last_fullsized":"2024-04-26T20:02:03.239089+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:33.436022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239288+0000","last_change":"2024-04-26T20:02:02.167054+0000","last_active":"2024-04-26T20:02:03.239288+0000","last_peered":"2024-04-26T20:02:03.239288+0000","last_clean":"2024-04-26T20:02:03.239288+0000","last_became_active":"2024-04-26T20:02:02.166210+0000","last_became_peered":"2024-04-26T20:02:02.166210+0000","last_unstale":"2024-04-26T20:02:03.239288+0000","last_undegraded":"2024-04-26T20:02:03.239288+0000","last_fullsized":"2024-04-26T20:02:03.239288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236507+0000","last_change":"2024-04-26T20:02:02.140498+0000","last_active":"2024-04-26T20:02:03.236507+0000","last_peered":"2024-04-26T20:02:03.236507+0000","last_clean":"2024-04-26T20:02:03.236507+0000","last_became_active":"2024-04-26T20:02:02.140222+0000","last_became_peered":"2024-04-26T20:02:02.140222+0000","last_unstale":"2024-04-26T20:02:03.236507+0000","last_undegraded":"2024-04-26T20:02:03.236507+0000","last_fullsized":"2024-04-26T20:02:03.236507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:49.790277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237151+0000","last_change":"2024-04-26T20:02:02.141213+0000","last_active":"2024-04-26T20:02:03.237151+0000","last_peered":"2024-04-26T20:02:03.237151+0000","last_clean":"2024-04-26T20:02:03.237151+0000","last_became_active":"2024-04-26T20:02:02.140950+0000","last_became_peered":"2024-04-26T20:02:02.140950+0000","last_unstale":"2024-04-26T20:02:03.237151+0000","last_undegraded":"2024-04-26T20:02:03.237151+0000","last_fullsized":"2024-04-26T20:02:03.237151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:10.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238997+0000","last_change":"2024-04-26T20:02:02.165402+0000","last_active":"2024-04-26T20:02:03.238997+0000","last_peered":"2024-04-26T20:02:03.238997+0000","last_clean":"2024-04-26T20:02:03.238997+0000","last_became_active":"2024-04-26T20:02:02.165039+0000","last_became_peered":"2024-04-26T20:02:02.165039+0000","last_unstale":"2024-04-26T20:02:03.238997+0000","last_undegraded":"2024-04-26T20:02:03.238997+0000","last_fullsized":"2024-04-26T20:02:03.238997+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235925+0000","last_change":"2024-04-26T20:02:02.146950+0000","last_active":"2024-04-26T20:02:03.235925+0000","last_peered":"2024-04-26T20:02:03.235925+0000","last_clean":"2024-04-26T20:02:03.235925+0000","last_became_active":"2024-04-26T20:02:02.146326+0000","last_became_peered":"2024-04-26T20:02:02.146326+0000","last_unstale":"2024-04-26T20:02:03.235925+0000","last_undegraded":"2024-04-26T20:02:03.235925+0000","last_fullsized":"2024-04-26T20:02:03.235925+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:43.597063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239874+0000","last_change":"2024-04-26T20:02:02.157997+0000","last_active":"2024-04-26T20:02:03.239874+0000","last_peered":"2024-04-26T20:02:03.239874+0000","last_clean":"2024-04-26T20:02:03.239874+0000","last_became_active":"2024-04-26T20:02:02.155921+0000","last_became_peered":"2024-04-26T20:02:02.155921+0000","last_unstale":"2024-04-26T20:02:03.239874+0000","last_undegraded":"2024-04-26T20:02:03.239874+0000","last_fullsized":"2024-04-26T20:02:03.239874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:20.786310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149079+0000","last_change":"2024-04-26T20:02:02.160986+0000","last_active":"2024-04-26T20:02:03.149079+0000","last_peered":"2024-04-26T20:02:03.149079+0000","last_clean":"2024-04-26T20:02:03.149079+0000","last_became_active":"2024-04-26T20:02:02.160693+0000","last_became_peered":"2024-04-26T20:02:02.160693+0000","last_unstale":"2024-04-26T20:02:03.149079+0000","last_undegraded":"2024-04-26T20:02:03.149079+0000","last_fullsized":"2024-04-26T20:02:03.149079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:42.992497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236630+0000","last_change":"2024-04-26T20:02:02.149529+0000","last_active":"2024-04-26T20:02:03.236630+0000","last_peered":"2024-04-26T20:02:03.236630+0000","last_clean":"2024-04-26T20:02:03.236630+0000","last_became_active":"2024-04-26T20:02:02.148380+0000","last_became_peered":"2024-04-26T20:02:02.148380+0000","last_unstale":"2024-04-26T20:02:03.236630+0000","last_undegraded":"2024-04-26T20:02:03.236630+0000","last_fullsized":"2024-04-26T20:02:03.236630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:41.211746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237736+0000","last_change":"2024-04-26T20:02:02.157317+0000","last_active":"2024-04-26T20:02:03.237736+0000","last_peered":"2024-04-26T20:02:03.237736+0000","last_clean":"2024-04-26T20:02:03.237736+0000","last_became_active":"2024-04-26T20:02:02.157063+0000","last_became_peered":"2024-04-26T20:02:02.157063+0000","last_unstale":"2024-04-26T20:02:03.237736+0000","last_undegraded":"2024-04-26T20:02:03.237736+0000","last_fullsized":"2024-04-26T20:02:03.237736+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:18.938594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237967+0000","last_change":"2024-04-26T20:02:02.166022+0000","last_active":"2024-04-26T20:02:03.237967+0000","last_peered":"2024-04-26T20:02:03.237967+0000","last_clean":"2024-04-26T20:02:03.237967+0000","last_became_active":"2024-04-26T20:02:02.165717+0000","last_became_peered":"2024-04-26T20:02:02.165717+0000","last_unstale":"2024-04-26T20:02:03.237967+0000","last_undegraded":"2024-04-26T20:02:03.237967+0000","last_fullsized":"2024-04-26T20:02:03.237967+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236343+0000","last_change":"2024-04-26T20:02:02.159795+0000","last_active":"2024-04-26T20:02:03.236343+0000","last_peered":"2024-04-26T20:02:03.236343+0000","last_clean":"2024-04-26T20:02:03.236343+0000","last_became_active":"2024-04-26T20:02:02.159453+0000","last_became_peered":"2024-04-26T20:02:02.159453+0000","last_unstale":"2024-04-26T20:02:03.236343+0000","last_undegraded":"2024-04-26T20:02:03.236343+0000","last_fullsized":"2024-04-26T20:02:03.236343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:13.456162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237886+0000","last_change":"2024-04-26T20:02:02.164919+0000","last_active":"2024-04-26T20:02:03.237886+0000","last_peered":"2024-04-26T20:02:03.237886+0000","last_clean":"2024-04-26T20:02:03.237886+0000","last_became_active":"2024-04-26T20:02:02.164660+0000","last_became_peered":"2024-04-26T20:02:02.164660+0000","last_unstale":"2024-04-26T20:02:03.237886+0000","last_undegraded":"2024-04-26T20:02:03.237886+0000","last_fullsized":"2024-04-26T20:02:03.237886+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:34.099908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238768+0000","last_change":"2024-04-26T20:02:02.165762+0000","last_active":"2024-04-26T20:02:03.238768+0000","last_peered":"2024-04-26T20:02:03.238768+0000","last_clean":"2024-04-26T20:02:03.238768+0000","last_became_active":"2024-04-26T20:02:02.165478+0000","last_became_peered":"2024-04-26T20:02:02.165478+0000","last_unstale":"2024-04-26T20:02:03.238768+0000","last_undegraded":"2024-04-26T20:02:03.238768+0000","last_fullsized":"2024-04-26T20:02:03.238768+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235608+0000","last_change":"2024-04-26T20:02:02.143180+0000","last_active":"2024-04-26T20:02:03.235608+0000","last_peered":"2024-04-26T20:02:03.235608+0000","last_clean":"2024-04-26T20:02:03.235608+0000","last_became_active":"2024-04-26T20:02:02.138046+0000","last_became_peered":"2024-04-26T20:02:02.138046+0000","last_unstale":"2024-04-26T20:02:03.235608+0000","last_undegraded":"2024-04-26T20:02:03.235608+0000","last_fullsized":"2024-04-26T20:02:03.235608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:53.041114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237793+0000","last_change":"2024-04-26T20:02:02.149273+0000","last_active":"2024-04-26T20:02:03.237793+0000","last_peered":"2024-04-26T20:02:03.237793+0000","last_clean":"2024-04-26T20:02:03.237793+0000","last_became_active":"2024-04-26T20:02:02.148981+0000","last_became_peered":"2024-04-26T20:02:02.148981+0000","last_unstale":"2024-04-26T20:02:03.237793+0000","last_undegraded":"2024-04-26T20:02:03.237793+0000","last_fullsized":"2024-04-26T20:02:03.237793+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238974+0000","last_change":"2024-04-26T20:02:02.163204+0000","last_active":"2024-04-26T20:02:03.238974+0000","last_peered":"2024-04-26T20:02:03.238974+0000","last_clean":"2024-04-26T20:02:03.238974+0000","last_became_active":"2024-04-26T20:02:02.162514+0000","last_became_peered":"2024-04-26T20:02:02.162514+0000","last_unstale":"2024-04-26T20:02:03.238974+0000","last_undegraded":"2024-04-26T20:02:03.238974+0000","last_fullsized":"2024-04-26T20:02:03.238974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239386+0000","last_change":"2024-04-26T20:02:01.144979+0000","last_active":"2024-04-26T20:02:03.239386+0000","last_peered":"2024-04-26T20:02:03.239386+0000","last_clean":"2024-04-26T20:02:03.239386+0000","last_became_active":"2024-04-26T20:02:01.144650+0000","last_became_peered":"2024-04-26T20:02:01.144650+0000","last_unstale":"2024-04-26T20:02:03.239386+0000","last_undegraded":"2024-04-26T20:02:03.239386+0000","last_fullsized":"2024-04-26T20:02:03.239386+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147523+0000","last_change":"2024-04-26T20:02:02.155432+0000","last_active":"2024-04-26T20:02:03.147523+0000","last_peered":"2024-04-26T20:02:03.147523+0000","last_clean":"2024-04-26T20:02:03.147523+0000","last_became_active":"2024-04-26T20:02:02.153246+0000","last_became_peered":"2024-04-26T20:02:02.153246+0000","last_unstale":"2024-04-26T20:02:03.147523+0000","last_undegraded":"2024-04-26T20:02:03.147523+0000","last_fullsized":"2024-04-26T20:02:03.147523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:31.583720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239186+0000","last_change":"2024-04-26T20:02:01.149903+0000","last_active":"2024-04-26T20:02:03.239186+0000","last_peered":"2024-04-26T20:02:03.239186+0000","last_clean":"2024-04-26T20:02:03.239186+0000","last_became_active":"2024-04-26T20:02:01.149641+0000","last_became_peered":"2024-04-26T20:02:01.149641+0000","last_unstale":"2024-04-26T20:02:03.239186+0000","last_undegraded":"2024-04-26T20:02:03.239186+0000","last_fullsized":"2024-04-26T20:02:03.239186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:25.390334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238151+0000","last_change":"2024-04-26T20:02:02.149195+0000","last_active":"2024-04-26T20:02:03.238151+0000","last_peered":"2024-04-26T20:02:03.238151+0000","last_clean":"2024-04-26T20:02:03.238151+0000","last_became_active":"2024-04-26T20:02:02.148929+0000","last_became_peered":"2024-04-26T20:02:02.148929+0000","last_unstale":"2024-04-26T20:02:03.238151+0000","last_undegraded":"2024-04-26T20:02:03.238151+0000","last_fullsized":"2024-04-26T20:02:03.238151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238738+0000","last_change":"2024-04-26T20:02:01.148651+0000","last_active":"2024-04-26T20:02:03.238738+0000","last_peered":"2024-04-26T20:02:03.238738+0000","last_clean":"2024-04-26T20:02:03.238738+0000","last_became_active":"2024-04-26T20:02:01.148343+0000","last_became_peered":"2024-04-26T20:02:01.148343+0000","last_unstale":"2024-04-26T20:02:03.238738+0000","last_undegraded":"2024-04-26T20:02:03.238738+0000","last_fullsized":"2024-04-26T20:02:03.238738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238255+0000","last_change":"2024-04-26T20:02:02.161891+0000","last_active":"2024-04-26T20:02:03.238255+0000","last_peered":"2024-04-26T20:02:03.238255+0000","last_clean":"2024-04-26T20:02:03.238255+0000","last_became_active":"2024-04-26T20:02:02.161338+0000","last_became_peered":"2024-04-26T20:02:02.161338+0000","last_unstale":"2024-04-26T20:02:03.238255+0000","last_undegraded":"2024-04-26T20:02:03.238255+0000","last_fullsized":"2024-04-26T20:02:03.238255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237809+0000","last_change":"2024-04-26T20:02:01.148830+0000","last_active":"2024-04-26T20:02:03.237809+0000","last_peered":"2024-04-26T20:02:03.237809+0000","last_clean":"2024-04-26T20:02:03.237809+0000","last_became_active":"2024-04-26T20:02:01.148527+0000","last_became_peered":"2024-04-26T20:02:01.148527+0000","last_unstale":"2024-04-26T20:02:03.237809+0000","last_undegraded":"2024-04-26T20:02:03.237809+0000","last_fullsized":"2024-04-26T20:02:03.237809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:40.005357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236945+0000","last_change":"2024-04-26T20:02:02.147161+0000","last_active":"2024-04-26T20:02:03.236945+0000","last_peered":"2024-04-26T20:02:03.236945+0000","last_clean":"2024-04-26T20:02:03.236945+0000","last_became_active":"2024-04-26T20:02:02.146700+0000","last_became_peered":"2024-04-26T20:02:02.146700+0000","last_unstale":"2024-04-26T20:02:03.236945+0000","last_undegraded":"2024-04-26T20:02:03.236945+0000","last_fullsized":"2024-04-26T20:02:03.236945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.083746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149173+0000","last_change":"2024-04-26T20:02:01.147048+0000","last_active":"2024-04-26T20:02:03.149173+0000","last_peered":"2024-04-26T20:02:03.149173+0000","last_clean":"2024-04-26T20:02:03.149173+0000","last_became_active":"2024-04-26T20:02:01.146474+0000","last_became_peered":"2024-04-26T20:02:01.146474+0000","last_unstale":"2024-04-26T20:02:03.149173+0000","last_undegraded":"2024-04-26T20:02:03.149173+0000","last_fullsized":"2024-04-26T20:02:03.149173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:01.504200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237474+0000","last_change":"2024-04-26T20:02:02.143048+0000","last_active":"2024-04-26T20:02:03.237474+0000","last_peered":"2024-04-26T20:02:03.237474+0000","last_clean":"2024-04-26T20:02:03.237474+0000","last_became_active":"2024-04-26T20:02:02.142774+0000","last_became_peered":"2024-04-26T20:02:02.142774+0000","last_unstale":"2024-04-26T20:02:03.237474+0000","last_undegraded":"2024-04-26T20:02:03.237474+0000","last_fullsized":"2024-04-26T20:02:03.237474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:15.594485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238274+0000","last_change":"2024-04-26T20:02:01.147107+0000","last_active":"2024-04-26T20:02:03.238274+0000","last_peered":"2024-04-26T20:02:03.238274+0000","last_clean":"2024-04-26T20:02:03.238274+0000","last_became_active":"2024-04-26T20:02:01.146850+0000","last_became_peered":"2024-04-26T20:02:01.146850+0000","last_unstale":"2024-04-26T20:02:03.238274+0000","last_undegraded":"2024-04-26T20:02:03.238274+0000","last_fullsized":"2024-04-26T20:02:03.238274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:59.157695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149081+0000","last_change":"2024-04-26T20:02:02.166936+0000","last_active":"2024-04-26T20:02:03.149081+0000","last_peered":"2024-04-26T20:02:03.149081+0000","last_clean":"2024-04-26T20:02:03.149081+0000","last_became_active":"2024-04-26T20:02:02.166612+0000","last_became_peered":"2024-04-26T20:02:02.166612+0000","last_unstale":"2024-04-26T20:02:03.149081+0000","last_undegraded":"2024-04-26T20:02:03.149081+0000","last_fullsized":"2024-04-26T20:02:03.149081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:54.604807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236393+0000","last_change":"2024-04-26T20:02:01.145719+0000","last_active":"2024-04-26T20:02:03.236393+0000","last_peered":"2024-04-26T20:02:03.236393+0000","last_clean":"2024-04-26T20:02:03.236393+0000","last_became_active":"2024-04-26T20:02:01.145254+0000","last_became_peered":"2024-04-26T20:02:01.145254+0000","last_unstale":"2024-04-26T20:02:03.236393+0000","last_undegraded":"2024-04-26T20:02:03.236393+0000","last_fullsized":"2024-04-26T20:02:03.236393+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236081+0000","last_change":"2024-04-26T20:02:02.162161+0000","last_active":"2024-04-26T20:02:03.236081+0000","last_peered":"2024-04-26T20:02:03.236081+0000","last_clean":"2024-04-26T20:02:03.236081+0000","last_became_active":"2024-04-26T20:02:02.159842+0000","last_became_peered":"2024-04-26T20:02:02.159842+0000","last_unstale":"2024-04-26T20:02:03.236081+0000","last_undegraded":"2024-04-26T20:02:03.236081+0000","last_fullsized":"2024-04-26T20:02:03.236081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:01.432928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149634+0000","last_change":"2024-04-26T20:02:01.140193+0000","last_active":"2024-04-26T20:02:03.149634+0000","last_peered":"2024-04-26T20:02:03.149634+0000","last_clean":"2024-04-26T20:02:03.149634+0000","last_became_active":"2024-04-26T20:02:01.139881+0000","last_became_peered":"2024-04-26T20:02:01.139881+0000","last_unstale":"2024-04-26T20:02:03.149634+0000","last_undegraded":"2024-04-26T20:02:03.149634+0000","last_fullsized":"2024-04-26T20:02:03.149634+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:24.798986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150274+0000","last_change":"2024-04-26T20:02:02.142384+0000","last_active":"2024-04-26T20:02:03.150274+0000","last_peered":"2024-04-26T20:02:03.150274+0000","last_clean":"2024-04-26T20:02:03.150274+0000","last_became_active":"2024-04-26T20:02:02.142101+0000","last_became_peered":"2024-04-26T20:02:02.142101+0000","last_unstale":"2024-04-26T20:02:03.150274+0000","last_undegraded":"2024-04-26T20:02:03.150274+0000","last_fullsized":"2024-04-26T20:02:03.150274+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:53.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240368+0000","last_change":"2024-04-26T20:02:01.157641+0000","last_active":"2024-04-26T20:02:03.240368+0000","last_peered":"2024-04-26T20:02:03.240368+0000","last_clean":"2024-04-26T20:02:03.240368+0000","last_became_active":"2024-04-26T20:02:01.157289+0000","last_became_peered":"2024-04-26T20:02:01.157289+0000","last_unstale":"2024-04-26T20:02:03.240368+0000","last_undegraded":"2024-04-26T20:02:03.240368+0000","last_fullsized":"2024-04-26T20:02:03.240368+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240208+0000","last_change":"2024-04-26T20:02:02.162233+0000","last_active":"2024-04-26T20:02:03.240208+0000","last_peered":"2024-04-26T20:02:03.240208+0000","last_clean":"2024-04-26T20:02:03.240208+0000","last_became_active":"2024-04-26T20:02:02.161524+0000","last_became_peered":"2024-04-26T20:02:02.161524+0000","last_unstale":"2024-04-26T20:02:03.240208+0000","last_undegraded":"2024-04-26T20:02:03.240208+0000","last_fullsized":"2024-04-26T20:02:03.240208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:12.313138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239429+0000","last_change":"2024-04-26T20:02:01.143055+0000","last_active":"2024-04-26T20:02:03.239429+0000","last_peered":"2024-04-26T20:02:03.239429+0000","last_clean":"2024-04-26T20:02:03.239429+0000","last_became_active":"2024-04-26T20:02:01.142764+0000","last_became_peered":"2024-04-26T20:02:01.142764+0000","last_unstale":"2024-04-26T20:02:03.239429+0000","last_undegraded":"2024-04-26T20:02:03.239429+0000","last_fullsized":"2024-04-26T20:02:03.239429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:38.014023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146434+0000","last_change":"2024-04-26T20:02:02.153895+0000","last_active":"2024-04-26T20:02:03.146434+0000","last_peered":"2024-04-26T20:02:03.146434+0000","last_clean":"2024-04-26T20:02:03.146434+0000","last_became_active":"2024-04-26T20:02:02.153646+0000","last_became_peered":"2024-04-26T20:02:02.153646+0000","last_unstale":"2024-04-26T20:02:03.146434+0000","last_undegraded":"2024-04-26T20:02:03.146434+0000","last_fullsized":"2024-04-26T20:02:03.146434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239961+0000","last_change":"2024-04-26T20:02:01.157279+0000","last_active":"2024-04-26T20:02:03.239961+0000","last_peered":"2024-04-26T20:02:03.239961+0000","last_clean":"2024-04-26T20:02:03.239961+0000","last_became_active":"2024-04-26T20:02:01.156862+0000","last_became_peered":"2024-04-26T20:02:01.156862+0000","last_unstale":"2024-04-26T20:02:03.239961+0000","last_undegraded":"2024-04-26T20:02:03.239961+0000","last_fullsized":"2024-04-26T20:02:03.239961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239540+0000","last_change":"2024-04-26T20:02:02.164666+0000","last_active":"2024-04-26T20:02:03.239540+0000","last_peered":"2024-04-26T20:02:03.239540+0000","last_clean":"2024-04-26T20:02:03.239540+0000","last_became_active":"2024-04-26T20:02:02.163858+0000","last_became_peered":"2024-04-26T20:02:02.163858+0000","last_unstale":"2024-04-26T20:02:03.239540+0000","last_undegraded":"2024-04-26T20:02:03.239540+0000","last_fullsized":"2024-04-26T20:02:03.239540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150354+0000","last_change":"2024-04-26T20:02:01.140072+0000","last_active":"2024-04-26T20:02:03.150354+0000","last_peered":"2024-04-26T20:02:03.150354+0000","last_clean":"2024-04-26T20:02:03.150354+0000","last_became_active":"2024-04-26T20:02:01.139748+0000","last_became_peered":"2024-04-26T20:02:01.139748+0000","last_unstale":"2024-04-26T20:02:03.150354+0000","last_undegraded":"2024-04-26T20:02:03.150354+0000","last_fullsized":"2024-04-26T20:02:03.150354+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236560+0000","last_change":"2024-04-26T20:02:02.162563+0000","last_active":"2024-04-26T20:02:03.236560+0000","last_peered":"2024-04-26T20:02:03.236560+0000","last_clean":"2024-04-26T20:02:03.236560+0000","last_became_active":"2024-04-26T20:02:02.161077+0000","last_became_peered":"2024-04-26T20:02:02.161077+0000","last_unstale":"2024-04-26T20:02:03.236560+0000","last_undegraded":"2024-04-26T20:02:03.236560+0000","last_fullsized":"2024-04-26T20:02:03.236560+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:57.134404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237650+0000","last_change":"2024-04-26T20:02:01.138772+0000","last_active":"2024-04-26T20:02:03.237650+0000","last_peered":"2024-04-26T20:02:03.237650+0000","last_clean":"2024-04-26T20:02:03.237650+0000","last_became_active":"2024-04-26T20:02:01.138304+0000","last_became_peered":"2024-04-26T20:02:01.138304+0000","last_unstale":"2024-04-26T20:02:03.237650+0000","last_undegraded":"2024-04-26T20:02:03.237650+0000","last_fullsized":"2024-04-26T20:02:03.237650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:31.876037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238949+0000","last_change":"2024-04-26T20:02:02.167078+0000","last_active":"2024-04-26T20:02:03.238949+0000","last_peered":"2024-04-26T20:02:03.238949+0000","last_clean":"2024-04-26T20:02:03.238949+0000","last_became_active":"2024-04-26T20:02:02.166744+0000","last_became_peered":"2024-04-26T20:02:02.166744+0000","last_unstale":"2024-04-26T20:02:03.238949+0000","last_undegraded":"2024-04-26T20:02:03.238949+0000","last_fullsized":"2024-04-26T20:02:03.238949+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150446+0000","last_change":"2024-04-26T20:02:01.146462+0000","last_active":"2024-04-26T20:02:03.150446+0000","last_peered":"2024-04-26T20:02:03.150446+0000","last_clean":"2024-04-26T20:02:03.150446+0000","last_became_active":"2024-04-26T20:02:01.144773+0000","last_became_peered":"2024-04-26T20:02:01.144773+0000","last_unstale":"2024-04-26T20:02:03.150446+0000","last_undegraded":"2024-04-26T20:02:03.150446+0000","last_fullsized":"2024-04-26T20:02:03.150446+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:31.491400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150047+0000","last_change":"2024-04-26T20:02:02.156782+0000","last_active":"2024-04-26T20:02:03.150047+0000","last_peered":"2024-04-26T20:02:03.150047+0000","last_clean":"2024-04-26T20:02:03.150047+0000","last_became_active":"2024-04-26T20:02:02.156432+0000","last_became_peered":"2024-04-26T20:02:02.156432+0000","last_unstale":"2024-04-26T20:02:03.150047+0000","last_undegraded":"2024-04-26T20:02:03.150047+0000","last_fullsized":"2024-04-26T20:02:03.150047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:53.796204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236547+0000","last_change":"2024-04-26T20:02:01.151256+0000","last_active":"2024-04-26T20:02:03.236547+0000","last_peered":"2024-04-26T20:02:03.236547+0000","last_clean":"2024-04-26T20:02:03.236547+0000","last_became_active":"2024-04-26T20:02:01.150923+0000","last_became_peered":"2024-04-26T20:02:01.150923+0000","last_unstale":"2024-04-26T20:02:03.236547+0000","last_undegraded":"2024-04-26T20:02:03.236547+0000","last_fullsized":"2024-04-26T20:02:03.236547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239686+0000","last_change":"2024-04-26T20:02:02.157104+0000","last_active":"2024-04-26T20:02:03.239686+0000","last_peered":"2024-04-26T20:02:03.239686+0000","last_clean":"2024-04-26T20:02:03.239686+0000","last_became_active":"2024-04-26T20:02:02.156832+0000","last_became_peered":"2024-04-26T20:02:02.156832+0000","last_unstale":"2024-04-26T20:02:03.239686+0000","last_undegraded":"2024-04-26T20:02:03.239686+0000","last_fullsized":"2024-04-26T20:02:03.239686+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238418+0000","last_change":"2024-04-26T20:02:01.153327+0000","last_active":"2024-04-26T20:02:03.238418+0000","last_peered":"2024-04-26T20:02:03.238418+0000","last_clean":"2024-04-26T20:02:03.238418+0000","last_became_active":"2024-04-26T20:02:01.152975+0000","last_became_peered":"2024-04-26T20:02:01.152975+0000","last_unstale":"2024-04-26T20:02:03.238418+0000","last_undegraded":"2024-04-26T20:02:03.238418+0000","last_fullsized":"2024-04-26T20:02:03.238418+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239726+0000","last_change":"2024-04-26T20:02:02.168360+0000","last_active":"2024-04-26T20:02:03.239726+0000","last_peered":"2024-04-26T20:02:03.239726+0000","last_clean":"2024-04-26T20:02:03.239726+0000","last_became_active":"2024-04-26T20:02:02.166272+0000","last_became_peered":"2024-04-26T20:02:02.166272+0000","last_unstale":"2024-04-26T20:02:03.239726+0000","last_undegraded":"2024-04-26T20:02:03.239726+0000","last_fullsized":"2024-04-26T20:02:03.239726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239268+0000","last_change":"2024-04-26T20:02:01.142280+0000","last_active":"2024-04-26T20:02:03.239268+0000","last_peered":"2024-04-26T20:02:03.239268+0000","last_clean":"2024-04-26T20:02:03.239268+0000","last_became_active":"2024-04-26T20:02:01.142004+0000","last_became_peered":"2024-04-26T20:02:01.142004+0000","last_unstale":"2024-04-26T20:02:03.239268+0000","last_undegraded":"2024-04-26T20:02:03.239268+0000","last_fullsized":"2024-04-26T20:02:03.239268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:28.587601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239134+0000","last_change":"2024-04-26T20:02:02.158922+0000","last_active":"2024-04-26T20:02:03.239134+0000","last_peered":"2024-04-26T20:02:03.239134+0000","last_clean":"2024-04-26T20:02:03.239134+0000","last_became_active":"2024-04-26T20:02:02.158537+0000","last_became_peered":"2024-04-26T20:02:02.158537+0000","last_unstale":"2024-04-26T20:02:03.239134+0000","last_undegraded":"2024-04-26T20:02:03.239134+0000","last_fullsized":"2024-04-26T20:02:03.239134+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239067+0000","last_change":"2024-04-26T20:02:01.140742+0000","last_active":"2024-04-26T20:02:03.239067+0000","last_peered":"2024-04-26T20:02:03.239067+0000","last_clean":"2024-04-26T20:02:03.239067+0000","last_became_active":"2024-04-26T20:02:01.140411+0000","last_became_peered":"2024-04-26T20:02:01.140411+0000","last_unstale":"2024-04-26T20:02:03.239067+0000","last_undegraded":"2024-04-26T20:02:03.239067+0000","last_fullsized":"2024-04-26T20:02:03.239067+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.158278+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.157881+0000","last_became_peered":"2024-04-26T20:02:02.157881+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:08.790750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237528+0000","last_change":"2024-04-26T20:02:01.148686+0000","last_active":"2024-04-26T20:02:03.237528+0000","last_peered":"2024-04-26T20:02:03.237528+0000","last_clean":"2024-04-26T20:02:03.237528+0000","last_became_active":"2024-04-26T20:02:01.148391+0000","last_became_peered":"2024-04-26T20:02:01.148391+0000","last_unstale":"2024-04-26T20:02:03.237528+0000","last_undegraded":"2024-04-26T20:02:03.237528+0000","last_fullsized":"2024-04-26T20:02:03.237528+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239820+0000","last_change":"2024-04-26T20:02:02.161096+0000","last_active":"2024-04-26T20:02:03.239820+0000","last_peered":"2024-04-26T20:02:03.239820+0000","last_clean":"2024-04-26T20:02:03.239820+0000","last_became_active":"2024-04-26T20:02:02.159993+0000","last_became_peered":"2024-04-26T20:02:02.159993+0000","last_unstale":"2024-04-26T20:02:03.239820+0000","last_undegraded":"2024-04-26T20:02:03.239820+0000","last_fullsized":"2024-04-26T20:02:03.239820+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239709+0000","last_change":"2024-04-26T20:02:01.145418+0000","last_active":"2024-04-26T20:02:03.239709+0000","last_peered":"2024-04-26T20:02:03.239709+0000","last_clean":"2024-04-26T20:02:03.239709+0000","last_became_active":"2024-04-26T20:02:01.145087+0000","last_became_peered":"2024-04-26T20:02:01.145087+0000","last_unstale":"2024-04-26T20:02:03.239709+0000","last_undegraded":"2024-04-26T20:02:03.239709+0000","last_fullsized":"2024-04-26T20:02:03.239709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236593+0000","last_change":"2024-04-26T20:02:02.154537+0000","last_active":"2024-04-26T20:02:03.236593+0000","last_peered":"2024-04-26T20:02:03.236593+0000","last_clean":"2024-04-26T20:02:03.236593+0000","last_became_active":"2024-04-26T20:02:02.154074+0000","last_became_peered":"2024-04-26T20:02:02.154074+0000","last_unstale":"2024-04-26T20:02:03.236593+0000","last_undegraded":"2024-04-26T20:02:03.236593+0000","last_fullsized":"2024-04-26T20:02:03.236593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:59.493711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238043+0000","last_change":"2024-04-26T20:02:01.138552+0000","last_active":"2024-04-26T20:02:03.238043+0000","last_peered":"2024-04-26T20:02:03.238043+0000","last_clean":"2024-04-26T20:02:03.238043+0000","last_became_active":"2024-04-26T20:02:01.137755+0000","last_became_peered":"2024-04-26T20:02:01.137755+0000","last_unstale":"2024-04-26T20:02:03.238043+0000","last_undegraded":"2024-04-26T20:02:03.238043+0000","last_fullsized":"2024-04-26T20:02:03.238043+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:29.366170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238605+0000","last_change":"2024-04-26T20:02:02.158357+0000","last_active":"2024-04-26T20:02:03.238605+0000","last_peered":"2024-04-26T20:02:03.238605+0000","last_clean":"2024-04-26T20:02:03.238605+0000","last_became_active":"2024-04-26T20:02:02.158099+0000","last_became_peered":"2024-04-26T20:02:02.158099+0000","last_unstale":"2024-04-26T20:02:03.238605+0000","last_undegraded":"2024-04-26T20:02:03.238605+0000","last_fullsized":"2024-04-26T20:02:03.238605+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146799+0000","last_change":"2024-04-26T20:02:01.141454+0000","last_active":"2024-04-26T20:02:03.146799+0000","last_peered":"2024-04-26T20:02:03.146799+0000","last_clean":"2024-04-26T20:02:03.146799+0000","last_became_active":"2024-04-26T20:02:01.141165+0000","last_became_peered":"2024-04-26T20:02:01.141165+0000","last_unstale":"2024-04-26T20:02:03.146799+0000","last_undegraded":"2024-04-26T20:02:03.146799+0000","last_fullsized":"2024-04-26T20:02:03.146799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:54.136944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237924+0000","last_change":"2024-04-26T20:02:02.164395+0000","last_active":"2024-04-26T20:02:03.237924+0000","last_peered":"2024-04-26T20:02:03.237924+0000","last_clean":"2024-04-26T20:02:03.237924+0000","last_became_active":"2024-04-26T20:02:02.163976+0000","last_became_peered":"2024-04-26T20:02:02.163976+0000","last_unstale":"2024-04-26T20:02:03.237924+0000","last_undegraded":"2024-04-26T20:02:03.237924+0000","last_fullsized":"2024-04-26T20:02:03.237924+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236377+0000","last_change":"2024-04-26T20:02:01.149337+0000","last_active":"2024-04-26T20:02:03.236377+0000","last_peered":"2024-04-26T20:02:03.236377+0000","last_clean":"2024-04-26T20:02:03.236377+0000","last_became_active":"2024-04-26T20:02:01.147128+0000","last_became_peered":"2024-04-26T20:02:01.147128+0000","last_unstale":"2024-04-26T20:02:03.236377+0000","last_undegraded":"2024-04-26T20:02:03.236377+0000","last_fullsized":"2024-04-26T20:02:03.236377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:50.270032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238079+0000","last_change":"2024-04-26T20:02:02.156728+0000","last_active":"2024-04-26T20:02:03.238079+0000","last_peered":"2024-04-26T20:02:03.238079+0000","last_clean":"2024-04-26T20:02:03.238079+0000","last_became_active":"2024-04-26T20:02:02.156400+0000","last_became_peered":"2024-04-26T20:02:02.156400+0000","last_unstale":"2024-04-26T20:02:03.238079+0000","last_undegraded":"2024-04-26T20:02:03.238079+0000","last_fullsized":"2024-04-26T20:02:03.238079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147793+0000","last_change":"2024-04-26T20:02:01.130730+0000","last_active":"2024-04-26T20:02:03.147793+0000","last_peered":"2024-04-26T20:02:03.147793+0000","last_clean":"2024-04-26T20:02:03.147793+0000","last_became_active":"2024-04-26T20:02:01.130411+0000","last_became_peered":"2024-04-26T20:02:01.130411+0000","last_unstale":"2024-04-26T20:02:03.147793+0000","last_undegraded":"2024-04-26T20:02:03.147793+0000","last_fullsized":"2024-04-26T20:02:03.147793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:15.214957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238085+0000","last_change":"2024-04-26T20:02:02.162955+0000","last_active":"2024-04-26T20:02:03.238085+0000","last_peered":"2024-04-26T20:02:03.238085+0000","last_clean":"2024-04-26T20:02:03.238085+0000","last_became_active":"2024-04-26T20:02:02.162676+0000","last_became_peered":"2024-04-26T20:02:02.162676+0000","last_unstale":"2024-04-26T20:02:03.238085+0000","last_undegraded":"2024-04-26T20:02:03.238085+0000","last_fullsized":"2024-04-26T20:02:03.238085+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:04.136652+0000","last_change":"2024-04-26T20:02:01.145916+0000","last_active":"2024-04-26T20:02:04.136652+0000","last_peered":"2024-04-26T20:02:04.136652+0000","last_clean":"2024-04-26T20:02:04.136652+0000","last_became_active":"2024-04-26T20:02:01.145659+0000","last_became_peered":"2024-04-26T20:02:01.145659+0000","last_unstale":"2024-04-26T20:02:04.136652+0000","last_undegraded":"2024-04-26T20:02:04.136652+0000","last_fullsized":"2024-04-26T20:02:04.136652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:32.895072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146590+0000","last_change":"2024-04-26T20:02:02.148305+0000","last_active":"2024-04-26T20:02:03.146590+0000","last_peered":"2024-04-26T20:02:03.146590+0000","last_clean":"2024-04-26T20:02:03.146590+0000","last_became_active":"2024-04-26T20:02:02.148103+0000","last_became_peered":"2024-04-26T20:02:02.148103+0000","last_unstale":"2024-04-26T20:02:03.146590+0000","last_undegraded":"2024-04-26T20:02:03.146590+0000","last_fullsized":"2024-04-26T20:02:03.146590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:00.173536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151161+0000","last_change":"2024-04-26T20:02:01.143779+0000","last_active":"2024-04-26T20:02:03.151161+0000","last_peered":"2024-04-26T20:02:03.151161+0000","last_clean":"2024-04-26T20:02:03.151161+0000","last_became_active":"2024-04-26T20:02:01.143448+0000","last_became_peered":"2024-04-26T20:02:01.143448+0000","last_unstale":"2024-04-26T20:02:03.151161+0000","last_undegraded":"2024-04-26T20:02:03.151161+0000","last_fullsized":"2024-04-26T20:02:03.151161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:07:53.783522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238528+0000","last_change":"2024-04-26T20:02:02.161591+0000","last_active":"2024-04-26T20:02:03.238528+0000","last_peered":"2024-04-26T20:02:03.238528+0000","last_clean":"2024-04-26T20:02:03.238528+0000","last_became_active":"2024-04-26T20:02:02.160934+0000","last_became_peered":"2024-04-26T20:02:02.160934+0000","last_unstale":"2024-04-26T20:02:03.238528+0000","last_undegraded":"2024-04-26T20:02:03.238528+0000","last_fullsized":"2024-04-26T20:02:03.238528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237874+0000","last_change":"2024-04-26T20:02:01.145371+0000","last_active":"2024-04-26T20:02:03.237874+0000","last_peered":"2024-04-26T20:02:03.237874+0000","last_clean":"2024-04-26T20:02:03.237874+0000","last_became_active":"2024-04-26T20:02:01.145075+0000","last_became_peered":"2024-04-26T20:02:01.145075+0000","last_unstale":"2024-04-26T20:02:03.237874+0000","last_undegraded":"2024-04-26T20:02:03.237874+0000","last_fullsized":"2024-04-26T20:02:03.237874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.135333+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.135022+0000","last_became_peered":"2024-04-26T20:02:02.135022+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:17.197831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146631+0000","last_change":"2024-04-26T20:02:01.137606+0000","last_active":"2024-04-26T20:02:03.146631+0000","last_peered":"2024-04-26T20:02:03.146631+0000","last_clean":"2024-04-26T20:02:03.146631+0000","last_became_active":"2024-04-26T20:02:01.137263+0000","last_became_peered":"2024-04-26T20:02:01.137263+0000","last_unstale":"2024-04-26T20:02:03.146631+0000","last_undegraded":"2024-04-26T20:02:03.146631+0000","last_fullsized":"2024-04-26T20:02:03.146631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:43.037846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239338+0000","last_change":"2024-04-26T20:02:02.166230+0000","last_active":"2024-04-26T20:02:03.239338+0000","last_peered":"2024-04-26T20:02:03.239338+0000","last_clean":"2024-04-26T20:02:03.239338+0000","last_became_active":"2024-04-26T20:02:02.160400+0000","last_became_peered":"2024-04-26T20:02:02.160400+0000","last_unstale":"2024-04-26T20:02:03.239338+0000","last_undegraded":"2024-04-26T20:02:03.239338+0000","last_fullsized":"2024-04-26T20:02:03.239338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149809+0000","last_change":"2024-04-26T20:02:01.146319+0000","last_active":"2024-04-26T20:02:03.149809+0000","last_peered":"2024-04-26T20:02:03.149809+0000","last_clean":"2024-04-26T20:02:03.149809+0000","last_became_active":"2024-04-26T20:02:01.145982+0000","last_became_peered":"2024-04-26T20:02:01.145982+0000","last_unstale":"2024-04-26T20:02:03.149809+0000","last_undegraded":"2024-04-26T20:02:03.149809+0000","last_fullsized":"2024-04-26T20:02:03.149809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:59.976511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146311+0000","last_change":"2024-04-26T20:02:02.161688+0000","last_active":"2024-04-26T20:02:03.146311+0000","last_peered":"2024-04-26T20:02:03.146311+0000","last_clean":"2024-04-26T20:02:03.146311+0000","last_became_active":"2024-04-26T20:02:02.159715+0000","last_became_peered":"2024-04-26T20:02:02.159715+0000","last_unstale":"2024-04-26T20:02:03.146311+0000","last_undegraded":"2024-04-26T20:02:03.146311+0000","last_fullsized":"2024-04-26T20:02:03.146311+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146475+0000","last_change":"2024-04-26T20:02:01.140827+0000","last_active":"2024-04-26T20:02:03.146475+0000","last_peered":"2024-04-26T20:02:03.146475+0000","last_clean":"2024-04-26T20:02:03.146475+0000","last_became_active":"2024-04-26T20:02:01.140496+0000","last_became_peered":"2024-04-26T20:02:01.140496+0000","last_unstale":"2024-04-26T20:02:03.146475+0000","last_undegraded":"2024-04-26T20:02:03.146475+0000","last_fullsized":"2024-04-26T20:02:03.146475+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236398+0000","last_change":"2024-04-26T20:02:02.163701+0000","last_active":"2024-04-26T20:02:03.236398+0000","last_peered":"2024-04-26T20:02:03.236398+0000","last_clean":"2024-04-26T20:02:03.236398+0000","last_became_active":"2024-04-26T20:02:02.162010+0000","last_became_peered":"2024-04-26T20:02:02.162010+0000","last_unstale":"2024-04-26T20:02:03.236398+0000","last_undegraded":"2024-04-26T20:02:03.236398+0000","last_fullsized":"2024-04-26T20:02:03.236398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:36.989382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235881+0000","last_change":"2024-04-26T20:02:01.142535+0000","last_active":"2024-04-26T20:02:03.235881+0000","last_peered":"2024-04-26T20:02:03.235881+0000","last_clean":"2024-04-26T20:02:03.235881+0000","last_became_active":"2024-04-26T20:02:01.142254+0000","last_became_peered":"2024-04-26T20:02:01.142254+0000","last_unstale":"2024-04-26T20:02:03.235881+0000","last_undegraded":"2024-04-26T20:02:03.235881+0000","last_fullsized":"2024-04-26T20:02:03.235881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:17.922754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235721+0000","last_change":"2024-04-26T20:02:02.142754+0000","last_active":"2024-04-26T20:02:03.235721+0000","last_peered":"2024-04-26T20:02:03.235721+0000","last_clean":"2024-04-26T20:02:03.235721+0000","last_became_active":"2024-04-26T20:02:02.142456+0000","last_became_peered":"2024-04-26T20:02:02.142456+0000","last_unstale":"2024-04-26T20:02:03.235721+0000","last_undegraded":"2024-04-26T20:02:03.235721+0000","last_fullsized":"2024-04-26T20:02:03.235721+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:39.370350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149884+0000","last_change":"2024-04-26T20:02:01.142278+0000","last_active":"2024-04-26T20:02:03.149884+0000","last_peered":"2024-04-26T20:02:03.149884+0000","last_clean":"2024-04-26T20:02:03.149884+0000","last_became_active":"2024-04-26T20:02:01.141891+0000","last_became_peered":"2024-04-26T20:02:01.141891+0000","last_unstale":"2024-04-26T20:02:03.149884+0000","last_undegraded":"2024-04-26T20:02:03.149884+0000","last_fullsized":"2024-04-26T20:02:03.149884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:27.768532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238359+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:02:03.238359+0000","last_peered":"2024-04-26T20:02:03.238359+0000","last_clean":"2024-04-26T20:02:03.238359+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:02:03.238359+0000","last_undegraded":"2024-04-26T20:02:03.238359+0000","last_fullsized":"2024-04-26T20:02:03.238359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":855448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":264920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":265572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":266532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":265010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:07.532 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:02:07.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.768+0000 7f6233200640 1 Processor -- start 2024-04-26T20:02:07.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.769+0000 7f6233200640 1 -- start start 2024-04-26T20:02:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6234073720 0x7f623406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f623415dec0 0x7f6234073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f623406ed40 0x7f62340cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f6234074040 con 0x7f6234073720 2024-04-26T20:02:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f62340701a0 con 0x7f623415dec0 2024-04-26T20:02:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.770+0000 7f6233200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f623415e320 con 0x7f623406ed40 2024-04-26T20:02:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6234073720 0x7f623406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f6234073720 0x7f623406fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57252/0 (socket says 172.21.15.88:57252) 2024-04-26T20:02:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 -- 172.21.15.88:0/209494121 learned_addr learned my addr 172.21.15.88:0/209494121 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231400640 1 --2- 172.21.15.88:0/209494121 >> v2:172.21.15.132:3300/0 conn(0x7f623415dec0 0x7f6234073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6232800640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f623406ed40 0x7f62340cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 -- 172.21.15.88:0/209494121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f623406ed40 msgr2=0x7f62340cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f623406ed40 0x7f62340cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 -- 172.21.15.88:0/209494121 >> v2:172.21.15.132:3300/0 conn(0x7f623415dec0 msgr2=0x7f6234073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 --2- 172.21.15.88:0/209494121 >> v2:172.21.15.132:3300/0 conn(0x7f623415dec0 0x7f6234073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.771+0000 7f6231e00640 1 -- 172.21.15.88:0/209494121 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623406df80 con 0x7f6234073720 2024-04-26T20:02:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.772+0000 7f6231e00640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6234073720 0x7f623406fa90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6234075ec0 tx=0x7f622402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7670d975fc9c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.773+0000 7f6230a00640 1 -- 172.21.15.88:0/209494121 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224037070 con 0x7f6234073720 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.773+0000 7f6230a00640 1 -- 172.21.15.88:0/209494121 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6224032440 con 0x7f6234073720 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.773+0000 7f6230a00640 1 -- 172.21.15.88:0/209494121 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224032810 con 0x7f6234073720 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.773+0000 7f6233200640 1 -- 172.21.15.88:0/209494121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6234073720 msgr2=0x7f623406fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.773+0000 7f6233200640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6234073720 0x7f623406fa90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6234075ec0 tx=0x7f622402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 -- 172.21.15.88:0/209494121 shutdown_connections 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f623406ed40 0x7f62340cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 --2- 172.21.15.88:0/209494121 >> v2:172.21.15.132:3300/0 conn(0x7f623415dec0 0x7f6234073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 --2- 172.21.15.88:0/209494121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6234073720 0x7f623406fa90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 -- 172.21.15.88:0/209494121 >> 172.21.15.88:0/209494121 conn(0x7f62340d0ae0 msgr2=0x7f62340d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 -- 172.21.15.88:0/209494121 shutdown_connections 2024-04-26T20:02:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.774+0000 7f6233200640 1 -- 172.21.15.88:0/209494121 wait complete. 2024-04-26T20:02:07.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.775+0000 7f6233200640 1 Processor -- start 2024-04-26T20:02:07.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.776+0000 7f6233200640 1 -- start start 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 0x7f6234135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6234073720 0x7f6234143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f623406f1f0 con 0x7f623415dec0 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f623406e540 con 0x7f623406ed40 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6233200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f6234146910 con 0x7f6234073720 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57266/0 (socket says 172.21.15.88:57266) 2024-04-26T20:02:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 -- 172.21.15.88:0/2759429812 learned_addr learned my addr 172.21.15.88:0/2759429812 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6231e00640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 0x7f6234135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6231400640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6234073720 0x7f6234143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 -- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6234073720 msgr2=0x7f6234143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6234073720 0x7f6234143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 -- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 msgr2=0x7f6234135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 0x7f6234135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6232800640 1 -- 172.21.15.88:0/2759429812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62340b4610 con 0x7f623415dec0 2024-04-26T20:02:07.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.777+0000 7f6231e00640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 0x7f6234135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f6232800640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f622c007e60 tx=0x7f622c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c014620 con 0x7f623415dec0 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f622c019070 con 0x7f623415dec0 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62340b48f0 con 0x7f623415dec0 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c014a20 con 0x7f623415dec0 2024-04-26T20:02:07.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.778+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6234135770 con 0x7f623415dec0 2024-04-26T20:02:07.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.779+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62340693c0 con 0x7f623415dec0 2024-04-26T20:02:07.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.780+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f622c014bc0 con 0x7f623415dec0 2024-04-26T20:02:07.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.781+0000 7f622a000640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 0x7f621006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:07.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.781+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f622c08c500 con 0x7f623415dec0 2024-04-26T20:02:07.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.781+0000 7f6231e00640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 0x7f621006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:07.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.782+0000 7f6231e00640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 0x7f621006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:07.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:07.786+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f622c05a800 con 0x7f623415dec0 2024-04-26T20:02:08.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.059+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6234144a00 con 0x7f623415dec0 2024-04-26T20:02:08.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.060+0000 7f622a000640 1 -- 172.21.15.88:0/2759429812 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1432 (secure 0 0 0) 0x7f622c05a100 con 0x7f623415dec0 2024-04-26T20:02:08.061 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:08.061 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":166,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1714161574,"num_in_osds":8,"osd_in_since":1714161565,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":231858176,"bytes_avail":772862255104,"bytes_total":773094113280,"write_bytes_sec":2383,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":29,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5712}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-26T20:01:28.970986+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 msgr2=0x7f621006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 0x7f621006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402e000 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 msgr2=0x7f6234143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f622c007e60 tx=0x7f622c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 shutdown_connections 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f623415dec0 0x7f6234143790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f621006c8f0 0x7f621006edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f6234073720 0x7f6234143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 --2- 172.21.15.88:0/2759429812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f623406ed40 0x7f6234135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 >> 172.21.15.88:0/2759429812 conn(0x7f62340d0ae0 msgr2=0x7f62340722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:08.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 shutdown_connections 2024-04-26T20:02:08.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.063+0000 7f6233200640 1 -- 172.21.15.88:0/2759429812 wait complete. 2024-04-26T20:02:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:02:08.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.272+0000 7f0387200640 1 Processor -- start 2024-04-26T20:02:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.273+0000 7f0387200640 1 -- start start 2024-04-26T20:02:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0388146680 0x7f03880b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f03880d0e70 0x7f03880d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0388160930 0x7f0388160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f038806e970 con 0x7f0388146680 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f03880d17f0 con 0x7f03880d0e70 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.274+0000 7f0387200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0388146a60 con 0x7f0388160930 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0388146680 0x7f03880b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0388146680 0x7f03880b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57272/0 (socket says 172.21.15.88:57272) 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 -- 172.21.15.88:0/1647912632 learned_addr learned my addr 172.21.15.88:0/1647912632 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f037fe00640 1 --2- 172.21.15.88:0/1647912632 >> v2:172.21.15.132:3300/0 conn(0x7f03880d0e70 0x7f03880d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0386800640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0388160930 0x7f0388160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 -- 172.21.15.88:0/1647912632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0388160930 msgr2=0x7f0388160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:08.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0388160930 0x7f0388160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 -- 172.21.15.88:0/1647912632 >> v2:172.21.15.132:3300/0 conn(0x7f03880d0e70 msgr2=0x7f03880d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:08.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 --2- 172.21.15.88:0/1647912632 >> v2:172.21.15.132:3300/0 conn(0x7f03880d0e70 0x7f03880d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.275+0000 7f0385e00640 1 -- 172.21.15.88:0/1647912632 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038806dd60 con 0x7f0388146680 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.276+0000 7f0385e00640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388146680 0x7f03880b5990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0388160890 tx=0x7f037802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0eb86964a00be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.276+0000 7f0385400640 1 -- 172.21.15.88:0/1647912632 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378037070 con 0x7f0388146680 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0385400640 1 -- 172.21.15.88:0/1647912632 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0378032440 con 0x7f0388146680 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0385400640 1 -- 172.21.15.88:0/1647912632 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378032810 con 0x7f0388146680 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 -- 172.21.15.88:0/1647912632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388146680 msgr2=0x7f03880b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388146680 0x7f03880b5990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0388160890 tx=0x7f037802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 -- 172.21.15.88:0/1647912632 shutdown_connections 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0388160930 0x7f0388160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 --2- 172.21.15.88:0/1647912632 >> v2:172.21.15.132:3300/0 conn(0x7f03880d0e70 0x7f03880d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 --2- 172.21.15.88:0/1647912632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388146680 0x7f03880b5990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 -- 172.21.15.88:0/1647912632 >> 172.21.15.88:0/1647912632 conn(0x7f038806c740 msgr2=0x7f038806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 -- 172.21.15.88:0/1647912632 shutdown_connections 2024-04-26T20:02:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.277+0000 7f0387200640 1 -- 172.21.15.88:0/1647912632 wait complete. 2024-04-26T20:02:08.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.279+0000 7f0387200640 1 Processor -- start 2024-04-26T20:02:08.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.280+0000 7f0387200640 1 -- start start 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03880d0e70 0x7f03880caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 0x7f03880cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f038806e600 con 0x7f0388160930 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f038806e320 con 0x7f0388146680 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0387200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f03880cd6a0 con 0x7f03880d0e70 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0386800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0386800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57286/0 (socket says 172.21.15.88:57286) 2024-04-26T20:02:08.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0386800640 1 -- 172.21.15.88:0/3689132072 learned_addr learned my addr 172.21.15.88:0/3689132072 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:08.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f0385e00640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03880d0e70 0x7f03880caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.281+0000 7f037fe00640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 0x7f03880cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 -- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03880d0e70 msgr2=0x7f03880caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03880d0e70 0x7f03880caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 -- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 msgr2=0x7f03880cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 0x7f03880cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 -- 172.21.15.88:0/3689132072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0388142d00 con 0x7f0388160930 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f037fe00640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 0x7f03880cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0386800640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0380007e60 tx=0x7f038000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:08.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380014420 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0380019070 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380014820 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0388144160 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.282+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0388142a20 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.283+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03880693c0 con 0x7f0388160930 2024-04-26T20:02:08.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.285+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03800149c0 con 0x7f0388160930 2024-04-26T20:02:08.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.286+0000 7f037e000640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 0x7f036406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.286+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f038008c470 con 0x7f0388160930 2024-04-26T20:02:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.286+0000 7f0385e00640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 0x7f036406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.287+0000 7f0385e00640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 0x7f036406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0378034000 tx=0x7f037802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:08.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.290+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0388142d00 con 0x7f0388160930 2024-04-26T20:02:08.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.487+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03880b4ef0 con 0x7f036406c8f0 2024-04-26T20:02:08.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.494+0000 7f037e000640 1 -- 172.21.15.88:0/3689132072 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f03880b4ef0 con 0x7f036406c8f0 2024-04-26T20:02:08.498 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:08.499 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 msgr2=0x7f036406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 0x7f036406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0378034000 tx=0x7f037802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 msgr2=0x7f03880cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0380007e60 tx=0x7f038000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 shutdown_connections 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0388160930 0x7f03880cb970 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0388146680 0x7f03880cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f036406c8f0 0x7f036406edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 --2- 172.21.15.88:0/3689132072 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f03880d0e70 0x7f03880caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:08.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 >> 172.21.15.88:0/3689132072 conn(0x7f038806c740 msgr2=0x7f03880cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:08.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 shutdown_connections 2024-04-26T20:02:08.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:08.500+0000 7f0387200640 1 -- 172.21.15.88:0/3689132072 wait complete. 2024-04-26T20:02:08.519 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-26T20:02:06.993956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226424,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748296,"statfs":{"total":773094113280,"available":772862255104,"internally_reserved":0,"allocated":6234112,"data_stored":3304916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.014704"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236291+0000","last_change":"2024-04-26T20:02:02.143484+0000","last_active":"2024-04-26T20:02:03.236291+0000","last_peered":"2024-04-26T20:02:03.236291+0000","last_clean":"2024-04-26T20:02:03.236291+0000","last_became_active":"2024-04-26T20:02:02.143218+0000","last_became_peered":"2024-04-26T20:02:02.143218+0000","last_unstale":"2024-04-26T20:02:03.236291+0000","last_undegraded":"2024-04-26T20:02:03.236291+0000","last_fullsized":"2024-04-26T20:02:03.236291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:03.607368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238903+0000","last_change":"2024-04-26T20:02:02.137481+0000","last_active":"2024-04-26T20:02:03.238903+0000","last_peered":"2024-04-26T20:02:03.238903+0000","last_clean":"2024-04-26T20:02:03.238903+0000","last_became_active":"2024-04-26T20:02:02.136583+0000","last_became_peered":"2024-04-26T20:02:02.136583+0000","last_unstale":"2024-04-26T20:02:03.238903+0000","last_undegraded":"2024-04-26T20:02:03.238903+0000","last_fullsized":"2024-04-26T20:02:03.238903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:48.053920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239432+0000","last_change":"2024-04-26T20:02:02.168154+0000","last_active":"2024-04-26T20:02:03.239432+0000","last_peered":"2024-04-26T20:02:03.239432+0000","last_clean":"2024-04-26T20:02:03.239432+0000","last_became_active":"2024-04-26T20:02:02.167855+0000","last_became_peered":"2024-04-26T20:02:02.167855+0000","last_unstale":"2024-04-26T20:02:03.239432+0000","last_undegraded":"2024-04-26T20:02:03.239432+0000","last_fullsized":"2024-04-26T20:02:03.239432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237419+0000","last_change":"2024-04-26T20:02:02.157856+0000","last_active":"2024-04-26T20:02:03.237419+0000","last_peered":"2024-04-26T20:02:03.237419+0000","last_clean":"2024-04-26T20:02:03.237419+0000","last_became_active":"2024-04-26T20:02:02.156779+0000","last_became_peered":"2024-04-26T20:02:02.156779+0000","last_unstale":"2024-04-26T20:02:03.237419+0000","last_undegraded":"2024-04-26T20:02:03.237419+0000","last_fullsized":"2024-04-26T20:02:03.237419+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:26.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236036+0000","last_change":"2024-04-26T20:02:02.164743+0000","last_active":"2024-04-26T20:02:03.236036+0000","last_peered":"2024-04-26T20:02:03.236036+0000","last_clean":"2024-04-26T20:02:03.236036+0000","last_became_active":"2024-04-26T20:02:02.164403+0000","last_became_peered":"2024-04-26T20:02:02.164403+0000","last_unstale":"2024-04-26T20:02:03.236036+0000","last_undegraded":"2024-04-26T20:02:03.236036+0000","last_fullsized":"2024-04-26T20:02:03.236036+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:10.126360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238106+0000","last_change":"2024-04-26T20:02:02.153689+0000","last_active":"2024-04-26T20:02:03.238106+0000","last_peered":"2024-04-26T20:02:03.238106+0000","last_clean":"2024-04-26T20:02:03.238106+0000","last_became_active":"2024-04-26T20:02:02.153476+0000","last_became_peered":"2024-04-26T20:02:02.153476+0000","last_unstale":"2024-04-26T20:02:03.238106+0000","last_undegraded":"2024-04-26T20:02:03.238106+0000","last_fullsized":"2024-04-26T20:02:03.238106+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151473+0000","last_change":"2024-04-26T20:02:02.163749+0000","last_active":"2024-04-26T20:02:03.151473+0000","last_peered":"2024-04-26T20:02:03.151473+0000","last_clean":"2024-04-26T20:02:03.151473+0000","last_became_active":"2024-04-26T20:02:02.162498+0000","last_became_peered":"2024-04-26T20:02:02.162498+0000","last_unstale":"2024-04-26T20:02:03.151473+0000","last_undegraded":"2024-04-26T20:02:03.151473+0000","last_fullsized":"2024-04-26T20:02:03.151473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235764+0000","last_change":"2024-04-26T20:02:02.155433+0000","last_active":"2024-04-26T20:02:03.235764+0000","last_peered":"2024-04-26T20:02:03.235764+0000","last_clean":"2024-04-26T20:02:03.235764+0000","last_became_active":"2024-04-26T20:02:02.155167+0000","last_became_peered":"2024-04-26T20:02:02.155167+0000","last_unstale":"2024-04-26T20:02:03.235764+0000","last_undegraded":"2024-04-26T20:02:03.235764+0000","last_fullsized":"2024-04-26T20:02:03.235764+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149596+0000","last_change":"2024-04-26T20:02:02.145050+0000","last_active":"2024-04-26T20:02:03.149596+0000","last_peered":"2024-04-26T20:02:03.149596+0000","last_clean":"2024-04-26T20:02:03.149596+0000","last_became_active":"2024-04-26T20:02:02.144748+0000","last_became_peered":"2024-04-26T20:02:02.144748+0000","last_unstale":"2024-04-26T20:02:03.149596+0000","last_undegraded":"2024-04-26T20:02:03.149596+0000","last_fullsized":"2024-04-26T20:02:03.149596+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:27.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147062+0000","last_change":"2024-04-26T20:02:02.157240+0000","last_active":"2024-04-26T20:02:03.147062+0000","last_peered":"2024-04-26T20:02:03.147062+0000","last_clean":"2024-04-26T20:02:03.147062+0000","last_became_active":"2024-04-26T20:02:02.156963+0000","last_became_peered":"2024-04-26T20:02:02.156963+0000","last_unstale":"2024-04-26T20:02:03.147062+0000","last_undegraded":"2024-04-26T20:02:03.147062+0000","last_fullsized":"2024-04-26T20:02:03.147062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:01.598249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147030+0000","last_change":"2024-04-26T20:02:02.149796+0000","last_active":"2024-04-26T20:02:03.147030+0000","last_peered":"2024-04-26T20:02:03.147030+0000","last_clean":"2024-04-26T20:02:03.147030+0000","last_became_active":"2024-04-26T20:02:02.146559+0000","last_became_peered":"2024-04-26T20:02:02.146559+0000","last_unstale":"2024-04-26T20:02:03.147030+0000","last_undegraded":"2024-04-26T20:02:03.147030+0000","last_fullsized":"2024-04-26T20:02:03.147030+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:23.159614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237551+0000","last_change":"2024-04-26T20:02:02.163072+0000","last_active":"2024-04-26T20:02:03.237551+0000","last_peered":"2024-04-26T20:02:03.237551+0000","last_clean":"2024-04-26T20:02:03.237551+0000","last_became_active":"2024-04-26T20:02:02.162527+0000","last_became_peered":"2024-04-26T20:02:02.162527+0000","last_unstale":"2024-04-26T20:02:03.237551+0000","last_undegraded":"2024-04-26T20:02:03.237551+0000","last_fullsized":"2024-04-26T20:02:03.237551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:39.029864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151035+0000","last_change":"2024-04-26T20:02:02.165794+0000","last_active":"2024-04-26T20:02:03.151035+0000","last_peered":"2024-04-26T20:02:03.151035+0000","last_clean":"2024-04-26T20:02:03.151035+0000","last_became_active":"2024-04-26T20:02:02.165488+0000","last_became_peered":"2024-04-26T20:02:02.165488+0000","last_unstale":"2024-04-26T20:02:03.151035+0000","last_undegraded":"2024-04-26T20:02:03.151035+0000","last_fullsized":"2024-04-26T20:02:03.151035+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.541941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149361+0000","last_change":"2024-04-26T20:02:02.144523+0000","last_active":"2024-04-26T20:02:03.149361+0000","last_peered":"2024-04-26T20:02:03.149361+0000","last_clean":"2024-04-26T20:02:03.149361+0000","last_became_active":"2024-04-26T20:02:02.143456+0000","last_became_peered":"2024-04-26T20:02:02.143456+0000","last_unstale":"2024-04-26T20:02:03.149361+0000","last_undegraded":"2024-04-26T20:02:03.149361+0000","last_fullsized":"2024-04-26T20:02:03.149361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:41.021198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.152033+0000","last_change":"2024-04-26T20:02:02.141347+0000","last_active":"2024-04-26T20:02:03.152033+0000","last_peered":"2024-04-26T20:02:03.152033+0000","last_clean":"2024-04-26T20:02:03.152033+0000","last_became_active":"2024-04-26T20:02:02.141029+0000","last_became_peered":"2024-04-26T20:02:02.141029+0000","last_unstale":"2024-04-26T20:02:03.152033+0000","last_undegraded":"2024-04-26T20:02:03.152033+0000","last_fullsized":"2024-04-26T20:02:03.152033+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238361+0000","last_change":"2024-04-26T20:02:02.156328+0000","last_active":"2024-04-26T20:02:03.238361+0000","last_peered":"2024-04-26T20:02:03.238361+0000","last_clean":"2024-04-26T20:02:03.238361+0000","last_became_active":"2024-04-26T20:02:02.155610+0000","last_became_peered":"2024-04-26T20:02:02.155610+0000","last_unstale":"2024-04-26T20:02:03.238361+0000","last_undegraded":"2024-04-26T20:02:03.238361+0000","last_fullsized":"2024-04-26T20:02:03.238361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239089+0000","last_change":"2024-04-26T20:02:02.163223+0000","last_active":"2024-04-26T20:02:03.239089+0000","last_peered":"2024-04-26T20:02:03.239089+0000","last_clean":"2024-04-26T20:02:03.239089+0000","last_became_active":"2024-04-26T20:02:02.160865+0000","last_became_peered":"2024-04-26T20:02:02.160865+0000","last_unstale":"2024-04-26T20:02:03.239089+0000","last_undegraded":"2024-04-26T20:02:03.239089+0000","last_fullsized":"2024-04-26T20:02:03.239089+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:33.436022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239288+0000","last_change":"2024-04-26T20:02:02.167054+0000","last_active":"2024-04-26T20:02:03.239288+0000","last_peered":"2024-04-26T20:02:03.239288+0000","last_clean":"2024-04-26T20:02:03.239288+0000","last_became_active":"2024-04-26T20:02:02.166210+0000","last_became_peered":"2024-04-26T20:02:02.166210+0000","last_unstale":"2024-04-26T20:02:03.239288+0000","last_undegraded":"2024-04-26T20:02:03.239288+0000","last_fullsized":"2024-04-26T20:02:03.239288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236507+0000","last_change":"2024-04-26T20:02:02.140498+0000","last_active":"2024-04-26T20:02:03.236507+0000","last_peered":"2024-04-26T20:02:03.236507+0000","last_clean":"2024-04-26T20:02:03.236507+0000","last_became_active":"2024-04-26T20:02:02.140222+0000","last_became_peered":"2024-04-26T20:02:02.140222+0000","last_unstale":"2024-04-26T20:02:03.236507+0000","last_undegraded":"2024-04-26T20:02:03.236507+0000","last_fullsized":"2024-04-26T20:02:03.236507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:49.790277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237151+0000","last_change":"2024-04-26T20:02:02.141213+0000","last_active":"2024-04-26T20:02:03.237151+0000","last_peered":"2024-04-26T20:02:03.237151+0000","last_clean":"2024-04-26T20:02:03.237151+0000","last_became_active":"2024-04-26T20:02:02.140950+0000","last_became_peered":"2024-04-26T20:02:02.140950+0000","last_unstale":"2024-04-26T20:02:03.237151+0000","last_undegraded":"2024-04-26T20:02:03.237151+0000","last_fullsized":"2024-04-26T20:02:03.237151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:10.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238997+0000","last_change":"2024-04-26T20:02:02.165402+0000","last_active":"2024-04-26T20:02:03.238997+0000","last_peered":"2024-04-26T20:02:03.238997+0000","last_clean":"2024-04-26T20:02:03.238997+0000","last_became_active":"2024-04-26T20:02:02.165039+0000","last_became_peered":"2024-04-26T20:02:02.165039+0000","last_unstale":"2024-04-26T20:02:03.238997+0000","last_undegraded":"2024-04-26T20:02:03.238997+0000","last_fullsized":"2024-04-26T20:02:03.238997+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235925+0000","last_change":"2024-04-26T20:02:02.146950+0000","last_active":"2024-04-26T20:02:03.235925+0000","last_peered":"2024-04-26T20:02:03.235925+0000","last_clean":"2024-04-26T20:02:03.235925+0000","last_became_active":"2024-04-26T20:02:02.146326+0000","last_became_peered":"2024-04-26T20:02:02.146326+0000","last_unstale":"2024-04-26T20:02:03.235925+0000","last_undegraded":"2024-04-26T20:02:03.235925+0000","last_fullsized":"2024-04-26T20:02:03.235925+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:43.597063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239874+0000","last_change":"2024-04-26T20:02:02.157997+0000","last_active":"2024-04-26T20:02:03.239874+0000","last_peered":"2024-04-26T20:02:03.239874+0000","last_clean":"2024-04-26T20:02:03.239874+0000","last_became_active":"2024-04-26T20:02:02.155921+0000","last_became_peered":"2024-04-26T20:02:02.155921+0000","last_unstale":"2024-04-26T20:02:03.239874+0000","last_undegraded":"2024-04-26T20:02:03.239874+0000","last_fullsized":"2024-04-26T20:02:03.239874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:20.786310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149079+0000","last_change":"2024-04-26T20:02:02.160986+0000","last_active":"2024-04-26T20:02:03.149079+0000","last_peered":"2024-04-26T20:02:03.149079+0000","last_clean":"2024-04-26T20:02:03.149079+0000","last_became_active":"2024-04-26T20:02:02.160693+0000","last_became_peered":"2024-04-26T20:02:02.160693+0000","last_unstale":"2024-04-26T20:02:03.149079+0000","last_undegraded":"2024-04-26T20:02:03.149079+0000","last_fullsized":"2024-04-26T20:02:03.149079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:42.992497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236630+0000","last_change":"2024-04-26T20:02:02.149529+0000","last_active":"2024-04-26T20:02:03.236630+0000","last_peered":"2024-04-26T20:02:03.236630+0000","last_clean":"2024-04-26T20:02:03.236630+0000","last_became_active":"2024-04-26T20:02:02.148380+0000","last_became_peered":"2024-04-26T20:02:02.148380+0000","last_unstale":"2024-04-26T20:02:03.236630+0000","last_undegraded":"2024-04-26T20:02:03.236630+0000","last_fullsized":"2024-04-26T20:02:03.236630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:41.211746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237736+0000","last_change":"2024-04-26T20:02:02.157317+0000","last_active":"2024-04-26T20:02:03.237736+0000","last_peered":"2024-04-26T20:02:03.237736+0000","last_clean":"2024-04-26T20:02:03.237736+0000","last_became_active":"2024-04-26T20:02:02.157063+0000","last_became_peered":"2024-04-26T20:02:02.157063+0000","last_unstale":"2024-04-26T20:02:03.237736+0000","last_undegraded":"2024-04-26T20:02:03.237736+0000","last_fullsized":"2024-04-26T20:02:03.237736+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:18.938594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237967+0000","last_change":"2024-04-26T20:02:02.166022+0000","last_active":"2024-04-26T20:02:03.237967+0000","last_peered":"2024-04-26T20:02:03.237967+0000","last_clean":"2024-04-26T20:02:03.237967+0000","last_became_active":"2024-04-26T20:02:02.165717+0000","last_became_peered":"2024-04-26T20:02:02.165717+0000","last_unstale":"2024-04-26T20:02:03.237967+0000","last_undegraded":"2024-04-26T20:02:03.237967+0000","last_fullsized":"2024-04-26T20:02:03.237967+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236343+0000","last_change":"2024-04-26T20:02:02.159795+0000","last_active":"2024-04-26T20:02:03.236343+0000","last_peered":"2024-04-26T20:02:03.236343+0000","last_clean":"2024-04-26T20:02:03.236343+0000","last_became_active":"2024-04-26T20:02:02.159453+0000","last_became_peered":"2024-04-26T20:02:02.159453+0000","last_unstale":"2024-04-26T20:02:03.236343+0000","last_undegraded":"2024-04-26T20:02:03.236343+0000","last_fullsized":"2024-04-26T20:02:03.236343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:13.456162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237886+0000","last_change":"2024-04-26T20:02:02.164919+0000","last_active":"2024-04-26T20:02:03.237886+0000","last_peered":"2024-04-26T20:02:03.237886+0000","last_clean":"2024-04-26T20:02:03.237886+0000","last_became_active":"2024-04-26T20:02:02.164660+0000","last_became_peered":"2024-04-26T20:02:02.164660+0000","last_unstale":"2024-04-26T20:02:03.237886+0000","last_undegraded":"2024-04-26T20:02:03.237886+0000","last_fullsized":"2024-04-26T20:02:03.237886+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:34.099908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238768+0000","last_change":"2024-04-26T20:02:02.165762+0000","last_active":"2024-04-26T20:02:03.238768+0000","last_peered":"2024-04-26T20:02:03.238768+0000","last_clean":"2024-04-26T20:02:03.238768+0000","last_became_active":"2024-04-26T20:02:02.165478+0000","last_became_peered":"2024-04-26T20:02:02.165478+0000","last_unstale":"2024-04-26T20:02:03.238768+0000","last_undegraded":"2024-04-26T20:02:03.238768+0000","last_fullsized":"2024-04-26T20:02:03.238768+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235608+0000","last_change":"2024-04-26T20:02:02.143180+0000","last_active":"2024-04-26T20:02:03.235608+0000","last_peered":"2024-04-26T20:02:03.235608+0000","last_clean":"2024-04-26T20:02:03.235608+0000","last_became_active":"2024-04-26T20:02:02.138046+0000","last_became_peered":"2024-04-26T20:02:02.138046+0000","last_unstale":"2024-04-26T20:02:03.235608+0000","last_undegraded":"2024-04-26T20:02:03.235608+0000","last_fullsized":"2024-04-26T20:02:03.235608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:53.041114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237793+0000","last_change":"2024-04-26T20:02:02.149273+0000","last_active":"2024-04-26T20:02:03.237793+0000","last_peered":"2024-04-26T20:02:03.237793+0000","last_clean":"2024-04-26T20:02:03.237793+0000","last_became_active":"2024-04-26T20:02:02.148981+0000","last_became_peered":"2024-04-26T20:02:02.148981+0000","last_unstale":"2024-04-26T20:02:03.237793+0000","last_undegraded":"2024-04-26T20:02:03.237793+0000","last_fullsized":"2024-04-26T20:02:03.237793+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238974+0000","last_change":"2024-04-26T20:02:02.163204+0000","last_active":"2024-04-26T20:02:03.238974+0000","last_peered":"2024-04-26T20:02:03.238974+0000","last_clean":"2024-04-26T20:02:03.238974+0000","last_became_active":"2024-04-26T20:02:02.162514+0000","last_became_peered":"2024-04-26T20:02:02.162514+0000","last_unstale":"2024-04-26T20:02:03.238974+0000","last_undegraded":"2024-04-26T20:02:03.238974+0000","last_fullsized":"2024-04-26T20:02:03.238974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239386+0000","last_change":"2024-04-26T20:02:01.144979+0000","last_active":"2024-04-26T20:02:03.239386+0000","last_peered":"2024-04-26T20:02:03.239386+0000","last_clean":"2024-04-26T20:02:03.239386+0000","last_became_active":"2024-04-26T20:02:01.144650+0000","last_became_peered":"2024-04-26T20:02:01.144650+0000","last_unstale":"2024-04-26T20:02:03.239386+0000","last_undegraded":"2024-04-26T20:02:03.239386+0000","last_fullsized":"2024-04-26T20:02:03.239386+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147523+0000","last_change":"2024-04-26T20:02:02.155432+0000","last_active":"2024-04-26T20:02:03.147523+0000","last_peered":"2024-04-26T20:02:03.147523+0000","last_clean":"2024-04-26T20:02:03.147523+0000","last_became_active":"2024-04-26T20:02:02.153246+0000","last_became_peered":"2024-04-26T20:02:02.153246+0000","last_unstale":"2024-04-26T20:02:03.147523+0000","last_undegraded":"2024-04-26T20:02:03.147523+0000","last_fullsized":"2024-04-26T20:02:03.147523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:31.583720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239186+0000","last_change":"2024-04-26T20:02:01.149903+0000","last_active":"2024-04-26T20:02:03.239186+0000","last_peered":"2024-04-26T20:02:03.239186+0000","last_clean":"2024-04-26T20:02:03.239186+0000","last_became_active":"2024-04-26T20:02:01.149641+0000","last_became_peered":"2024-04-26T20:02:01.149641+0000","last_unstale":"2024-04-26T20:02:03.239186+0000","last_undegraded":"2024-04-26T20:02:03.239186+0000","last_fullsized":"2024-04-26T20:02:03.239186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:25.390334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238151+0000","last_change":"2024-04-26T20:02:02.149195+0000","last_active":"2024-04-26T20:02:03.238151+0000","last_peered":"2024-04-26T20:02:03.238151+0000","last_clean":"2024-04-26T20:02:03.238151+0000","last_became_active":"2024-04-26T20:02:02.148929+0000","last_became_peered":"2024-04-26T20:02:02.148929+0000","last_unstale":"2024-04-26T20:02:03.238151+0000","last_undegraded":"2024-04-26T20:02:03.238151+0000","last_fullsized":"2024-04-26T20:02:03.238151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238738+0000","last_change":"2024-04-26T20:02:01.148651+0000","last_active":"2024-04-26T20:02:03.238738+0000","last_peered":"2024-04-26T20:02:03.238738+0000","last_clean":"2024-04-26T20:02:03.238738+0000","last_became_active":"2024-04-26T20:02:01.148343+0000","last_became_peered":"2024-04-26T20:02:01.148343+0000","last_unstale":"2024-04-26T20:02:03.238738+0000","last_undegraded":"2024-04-26T20:02:03.238738+0000","last_fullsized":"2024-04-26T20:02:03.238738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238255+0000","last_change":"2024-04-26T20:02:02.161891+0000","last_active":"2024-04-26T20:02:03.238255+0000","last_peered":"2024-04-26T20:02:03.238255+0000","last_clean":"2024-04-26T20:02:03.238255+0000","last_became_active":"2024-04-26T20:02:02.161338+0000","last_became_peered":"2024-04-26T20:02:02.161338+0000","last_unstale":"2024-04-26T20:02:03.238255+0000","last_undegraded":"2024-04-26T20:02:03.238255+0000","last_fullsized":"2024-04-26T20:02:03.238255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237809+0000","last_change":"2024-04-26T20:02:01.148830+0000","last_active":"2024-04-26T20:02:03.237809+0000","last_peered":"2024-04-26T20:02:03.237809+0000","last_clean":"2024-04-26T20:02:03.237809+0000","last_became_active":"2024-04-26T20:02:01.148527+0000","last_became_peered":"2024-04-26T20:02:01.148527+0000","last_unstale":"2024-04-26T20:02:03.237809+0000","last_undegraded":"2024-04-26T20:02:03.237809+0000","last_fullsized":"2024-04-26T20:02:03.237809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:40.005357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236945+0000","last_change":"2024-04-26T20:02:02.147161+0000","last_active":"2024-04-26T20:02:03.236945+0000","last_peered":"2024-04-26T20:02:03.236945+0000","last_clean":"2024-04-26T20:02:03.236945+0000","last_became_active":"2024-04-26T20:02:02.146700+0000","last_became_peered":"2024-04-26T20:02:02.146700+0000","last_unstale":"2024-04-26T20:02:03.236945+0000","last_undegraded":"2024-04-26T20:02:03.236945+0000","last_fullsized":"2024-04-26T20:02:03.236945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.083746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149173+0000","last_change":"2024-04-26T20:02:01.147048+0000","last_active":"2024-04-26T20:02:03.149173+0000","last_peered":"2024-04-26T20:02:03.149173+0000","last_clean":"2024-04-26T20:02:03.149173+0000","last_became_active":"2024-04-26T20:02:01.146474+0000","last_became_peered":"2024-04-26T20:02:01.146474+0000","last_unstale":"2024-04-26T20:02:03.149173+0000","last_undegraded":"2024-04-26T20:02:03.149173+0000","last_fullsized":"2024-04-26T20:02:03.149173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:01.504200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237474+0000","last_change":"2024-04-26T20:02:02.143048+0000","last_active":"2024-04-26T20:02:03.237474+0000","last_peered":"2024-04-26T20:02:03.237474+0000","last_clean":"2024-04-26T20:02:03.237474+0000","last_became_active":"2024-04-26T20:02:02.142774+0000","last_became_peered":"2024-04-26T20:02:02.142774+0000","last_unstale":"2024-04-26T20:02:03.237474+0000","last_undegraded":"2024-04-26T20:02:03.237474+0000","last_fullsized":"2024-04-26T20:02:03.237474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:15.594485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238274+0000","last_change":"2024-04-26T20:02:01.147107+0000","last_active":"2024-04-26T20:02:03.238274+0000","last_peered":"2024-04-26T20:02:03.238274+0000","last_clean":"2024-04-26T20:02:03.238274+0000","last_became_active":"2024-04-26T20:02:01.146850+0000","last_became_peered":"2024-04-26T20:02:01.146850+0000","last_unstale":"2024-04-26T20:02:03.238274+0000","last_undegraded":"2024-04-26T20:02:03.238274+0000","last_fullsized":"2024-04-26T20:02:03.238274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:59.157695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149081+0000","last_change":"2024-04-26T20:02:02.166936+0000","last_active":"2024-04-26T20:02:03.149081+0000","last_peered":"2024-04-26T20:02:03.149081+0000","last_clean":"2024-04-26T20:02:03.149081+0000","last_became_active":"2024-04-26T20:02:02.166612+0000","last_became_peered":"2024-04-26T20:02:02.166612+0000","last_unstale":"2024-04-26T20:02:03.149081+0000","last_undegraded":"2024-04-26T20:02:03.149081+0000","last_fullsized":"2024-04-26T20:02:03.149081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:54.604807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236393+0000","last_change":"2024-04-26T20:02:01.145719+0000","last_active":"2024-04-26T20:02:03.236393+0000","last_peered":"2024-04-26T20:02:03.236393+0000","last_clean":"2024-04-26T20:02:03.236393+0000","last_became_active":"2024-04-26T20:02:01.145254+0000","last_became_peered":"2024-04-26T20:02:01.145254+0000","last_unstale":"2024-04-26T20:02:03.236393+0000","last_undegraded":"2024-04-26T20:02:03.236393+0000","last_fullsized":"2024-04-26T20:02:03.236393+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236081+0000","last_change":"2024-04-26T20:02:02.162161+0000","last_active":"2024-04-26T20:02:03.236081+0000","last_peered":"2024-04-26T20:02:03.236081+0000","last_clean":"2024-04-26T20:02:03.236081+0000","last_became_active":"2024-04-26T20:02:02.159842+0000","last_became_peered":"2024-04-26T20:02:02.159842+0000","last_unstale":"2024-04-26T20:02:03.236081+0000","last_undegraded":"2024-04-26T20:02:03.236081+0000","last_fullsized":"2024-04-26T20:02:03.236081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:01.432928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149634+0000","last_change":"2024-04-26T20:02:01.140193+0000","last_active":"2024-04-26T20:02:03.149634+0000","last_peered":"2024-04-26T20:02:03.149634+0000","last_clean":"2024-04-26T20:02:03.149634+0000","last_became_active":"2024-04-26T20:02:01.139881+0000","last_became_peered":"2024-04-26T20:02:01.139881+0000","last_unstale":"2024-04-26T20:02:03.149634+0000","last_undegraded":"2024-04-26T20:02:03.149634+0000","last_fullsized":"2024-04-26T20:02:03.149634+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:24.798986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150274+0000","last_change":"2024-04-26T20:02:02.142384+0000","last_active":"2024-04-26T20:02:03.150274+0000","last_peered":"2024-04-26T20:02:03.150274+0000","last_clean":"2024-04-26T20:02:03.150274+0000","last_became_active":"2024-04-26T20:02:02.142101+0000","last_became_peered":"2024-04-26T20:02:02.142101+0000","last_unstale":"2024-04-26T20:02:03.150274+0000","last_undegraded":"2024-04-26T20:02:03.150274+0000","last_fullsized":"2024-04-26T20:02:03.150274+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:53.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240368+0000","last_change":"2024-04-26T20:02:01.157641+0000","last_active":"2024-04-26T20:02:03.240368+0000","last_peered":"2024-04-26T20:02:03.240368+0000","last_clean":"2024-04-26T20:02:03.240368+0000","last_became_active":"2024-04-26T20:02:01.157289+0000","last_became_peered":"2024-04-26T20:02:01.157289+0000","last_unstale":"2024-04-26T20:02:03.240368+0000","last_undegraded":"2024-04-26T20:02:03.240368+0000","last_fullsized":"2024-04-26T20:02:03.240368+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240208+0000","last_change":"2024-04-26T20:02:02.162233+0000","last_active":"2024-04-26T20:02:03.240208+0000","last_peered":"2024-04-26T20:02:03.240208+0000","last_clean":"2024-04-26T20:02:03.240208+0000","last_became_active":"2024-04-26T20:02:02.161524+0000","last_became_peered":"2024-04-26T20:02:02.161524+0000","last_unstale":"2024-04-26T20:02:03.240208+0000","last_undegraded":"2024-04-26T20:02:03.240208+0000","last_fullsized":"2024-04-26T20:02:03.240208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:12.313138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239429+0000","last_change":"2024-04-26T20:02:01.143055+0000","last_active":"2024-04-26T20:02:03.239429+0000","last_peered":"2024-04-26T20:02:03.239429+0000","last_clean":"2024-04-26T20:02:03.239429+0000","last_became_active":"2024-04-26T20:02:01.142764+0000","last_became_peered":"2024-04-26T20:02:01.142764+0000","last_unstale":"2024-04-26T20:02:03.239429+0000","last_undegraded":"2024-04-26T20:02:03.239429+0000","last_fullsized":"2024-04-26T20:02:03.239429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:38.014023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146434+0000","last_change":"2024-04-26T20:02:02.153895+0000","last_active":"2024-04-26T20:02:03.146434+0000","last_peered":"2024-04-26T20:02:03.146434+0000","last_clean":"2024-04-26T20:02:03.146434+0000","last_became_active":"2024-04-26T20:02:02.153646+0000","last_became_peered":"2024-04-26T20:02:02.153646+0000","last_unstale":"2024-04-26T20:02:03.146434+0000","last_undegraded":"2024-04-26T20:02:03.146434+0000","last_fullsized":"2024-04-26T20:02:03.146434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239961+0000","last_change":"2024-04-26T20:02:01.157279+0000","last_active":"2024-04-26T20:02:03.239961+0000","last_peered":"2024-04-26T20:02:03.239961+0000","last_clean":"2024-04-26T20:02:03.239961+0000","last_became_active":"2024-04-26T20:02:01.156862+0000","last_became_peered":"2024-04-26T20:02:01.156862+0000","last_unstale":"2024-04-26T20:02:03.239961+0000","last_undegraded":"2024-04-26T20:02:03.239961+0000","last_fullsized":"2024-04-26T20:02:03.239961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239540+0000","last_change":"2024-04-26T20:02:02.164666+0000","last_active":"2024-04-26T20:02:03.239540+0000","last_peered":"2024-04-26T20:02:03.239540+0000","last_clean":"2024-04-26T20:02:03.239540+0000","last_became_active":"2024-04-26T20:02:02.163858+0000","last_became_peered":"2024-04-26T20:02:02.163858+0000","last_unstale":"2024-04-26T20:02:03.239540+0000","last_undegraded":"2024-04-26T20:02:03.239540+0000","last_fullsized":"2024-04-26T20:02:03.239540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150354+0000","last_change":"2024-04-26T20:02:01.140072+0000","last_active":"2024-04-26T20:02:03.150354+0000","last_peered":"2024-04-26T20:02:03.150354+0000","last_clean":"2024-04-26T20:02:03.150354+0000","last_became_active":"2024-04-26T20:02:01.139748+0000","last_became_peered":"2024-04-26T20:02:01.139748+0000","last_unstale":"2024-04-26T20:02:03.150354+0000","last_undegraded":"2024-04-26T20:02:03.150354+0000","last_fullsized":"2024-04-26T20:02:03.150354+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236560+0000","last_change":"2024-04-26T20:02:02.162563+0000","last_active":"2024-04-26T20:02:03.236560+0000","last_peered":"2024-04-26T20:02:03.236560+0000","last_clean":"2024-04-26T20:02:03.236560+0000","last_became_active":"2024-04-26T20:02:02.161077+0000","last_became_peered":"2024-04-26T20:02:02.161077+0000","last_unstale":"2024-04-26T20:02:03.236560+0000","last_undegraded":"2024-04-26T20:02:03.236560+0000","last_fullsized":"2024-04-26T20:02:03.236560+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:57.134404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237650+0000","last_change":"2024-04-26T20:02:01.138772+0000","last_active":"2024-04-26T20:02:03.237650+0000","last_peered":"2024-04-26T20:02:03.237650+0000","last_clean":"2024-04-26T20:02:03.237650+0000","last_became_active":"2024-04-26T20:02:01.138304+0000","last_became_peered":"2024-04-26T20:02:01.138304+0000","last_unstale":"2024-04-26T20:02:03.237650+0000","last_undegraded":"2024-04-26T20:02:03.237650+0000","last_fullsized":"2024-04-26T20:02:03.237650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:31.876037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238949+0000","last_change":"2024-04-26T20:02:02.167078+0000","last_active":"2024-04-26T20:02:03.238949+0000","last_peered":"2024-04-26T20:02:03.238949+0000","last_clean":"2024-04-26T20:02:03.238949+0000","last_became_active":"2024-04-26T20:02:02.166744+0000","last_became_peered":"2024-04-26T20:02:02.166744+0000","last_unstale":"2024-04-26T20:02:03.238949+0000","last_undegraded":"2024-04-26T20:02:03.238949+0000","last_fullsized":"2024-04-26T20:02:03.238949+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150446+0000","last_change":"2024-04-26T20:02:01.146462+0000","last_active":"2024-04-26T20:02:03.150446+0000","last_peered":"2024-04-26T20:02:03.150446+0000","last_clean":"2024-04-26T20:02:03.150446+0000","last_became_active":"2024-04-26T20:02:01.144773+0000","last_became_peered":"2024-04-26T20:02:01.144773+0000","last_unstale":"2024-04-26T20:02:03.150446+0000","last_undegraded":"2024-04-26T20:02:03.150446+0000","last_fullsized":"2024-04-26T20:02:03.150446+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:31.491400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150047+0000","last_change":"2024-04-26T20:02:02.156782+0000","last_active":"2024-04-26T20:02:03.150047+0000","last_peered":"2024-04-26T20:02:03.150047+0000","last_clean":"2024-04-26T20:02:03.150047+0000","last_became_active":"2024-04-26T20:02:02.156432+0000","last_became_peered":"2024-04-26T20:02:02.156432+0000","last_unstale":"2024-04-26T20:02:03.150047+0000","last_undegraded":"2024-04-26T20:02:03.150047+0000","last_fullsized":"2024-04-26T20:02:03.150047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:53.796204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236547+0000","last_change":"2024-04-26T20:02:01.151256+0000","last_active":"2024-04-26T20:02:03.236547+0000","last_peered":"2024-04-26T20:02:03.236547+0000","last_clean":"2024-04-26T20:02:03.236547+0000","last_became_active":"2024-04-26T20:02:01.150923+0000","last_became_peered":"2024-04-26T20:02:01.150923+0000","last_unstale":"2024-04-26T20:02:03.236547+0000","last_undegraded":"2024-04-26T20:02:03.236547+0000","last_fullsized":"2024-04-26T20:02:03.236547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239686+0000","last_change":"2024-04-26T20:02:02.157104+0000","last_active":"2024-04-26T20:02:03.239686+0000","last_peered":"2024-04-26T20:02:03.239686+0000","last_clean":"2024-04-26T20:02:03.239686+0000","last_became_active":"2024-04-26T20:02:02.156832+0000","last_became_peered":"2024-04-26T20:02:02.156832+0000","last_unstale":"2024-04-26T20:02:03.239686+0000","last_undegraded":"2024-04-26T20:02:03.239686+0000","last_fullsized":"2024-04-26T20:02:03.239686+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238418+0000","last_change":"2024-04-26T20:02:01.153327+0000","last_active":"2024-04-26T20:02:03.238418+0000","last_peered":"2024-04-26T20:02:03.238418+0000","last_clean":"2024-04-26T20:02:03.238418+0000","last_became_active":"2024-04-26T20:02:01.152975+0000","last_became_peered":"2024-04-26T20:02:01.152975+0000","last_unstale":"2024-04-26T20:02:03.238418+0000","last_undegraded":"2024-04-26T20:02:03.238418+0000","last_fullsized":"2024-04-26T20:02:03.238418+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239726+0000","last_change":"2024-04-26T20:02:02.168360+0000","last_active":"2024-04-26T20:02:03.239726+0000","last_peered":"2024-04-26T20:02:03.239726+0000","last_clean":"2024-04-26T20:02:03.239726+0000","last_became_active":"2024-04-26T20:02:02.166272+0000","last_became_peered":"2024-04-26T20:02:02.166272+0000","last_unstale":"2024-04-26T20:02:03.239726+0000","last_undegraded":"2024-04-26T20:02:03.239726+0000","last_fullsized":"2024-04-26T20:02:03.239726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239268+0000","last_change":"2024-04-26T20:02:01.142280+0000","last_active":"2024-04-26T20:02:03.239268+0000","last_peered":"2024-04-26T20:02:03.239268+0000","last_clean":"2024-04-26T20:02:03.239268+0000","last_became_active":"2024-04-26T20:02:01.142004+0000","last_became_peered":"2024-04-26T20:02:01.142004+0000","last_unstale":"2024-04-26T20:02:03.239268+0000","last_undegraded":"2024-04-26T20:02:03.239268+0000","last_fullsized":"2024-04-26T20:02:03.239268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:28.587601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239134+0000","last_change":"2024-04-26T20:02:02.158922+0000","last_active":"2024-04-26T20:02:03.239134+0000","last_peered":"2024-04-26T20:02:03.239134+0000","last_clean":"2024-04-26T20:02:03.239134+0000","last_became_active":"2024-04-26T20:02:02.158537+0000","last_became_peered":"2024-04-26T20:02:02.158537+0000","last_unstale":"2024-04-26T20:02:03.239134+0000","last_undegraded":"2024-04-26T20:02:03.239134+0000","last_fullsized":"2024-04-26T20:02:03.239134+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239067+0000","last_change":"2024-04-26T20:02:01.140742+0000","last_active":"2024-04-26T20:02:03.239067+0000","last_peered":"2024-04-26T20:02:03.239067+0000","last_clean":"2024-04-26T20:02:03.239067+0000","last_became_active":"2024-04-26T20:02:01.140411+0000","last_became_peered":"2024-04-26T20:02:01.140411+0000","last_unstale":"2024-04-26T20:02:03.239067+0000","last_undegraded":"2024-04-26T20:02:03.239067+0000","last_fullsized":"2024-04-26T20:02:03.239067+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.158278+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.157881+0000","last_became_peered":"2024-04-26T20:02:02.157881+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:08.790750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237528+0000","last_change":"2024-04-26T20:02:01.148686+0000","last_active":"2024-04-26T20:02:03.237528+0000","last_peered":"2024-04-26T20:02:03.237528+0000","last_clean":"2024-04-26T20:02:03.237528+0000","last_became_active":"2024-04-26T20:02:01.148391+0000","last_became_peered":"2024-04-26T20:02:01.148391+0000","last_unstale":"2024-04-26T20:02:03.237528+0000","last_undegraded":"2024-04-26T20:02:03.237528+0000","last_fullsized":"2024-04-26T20:02:03.237528+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239820+0000","last_change":"2024-04-26T20:02:02.161096+0000","last_active":"2024-04-26T20:02:03.239820+0000","last_peered":"2024-04-26T20:02:03.239820+0000","last_clean":"2024-04-26T20:02:03.239820+0000","last_became_active":"2024-04-26T20:02:02.159993+0000","last_became_peered":"2024-04-26T20:02:02.159993+0000","last_unstale":"2024-04-26T20:02:03.239820+0000","last_undegraded":"2024-04-26T20:02:03.239820+0000","last_fullsized":"2024-04-26T20:02:03.239820+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239709+0000","last_change":"2024-04-26T20:02:01.145418+0000","last_active":"2024-04-26T20:02:03.239709+0000","last_peered":"2024-04-26T20:02:03.239709+0000","last_clean":"2024-04-26T20:02:03.239709+0000","last_became_active":"2024-04-26T20:02:01.145087+0000","last_became_peered":"2024-04-26T20:02:01.145087+0000","last_unstale":"2024-04-26T20:02:03.239709+0000","last_undegraded":"2024-04-26T20:02:03.239709+0000","last_fullsized":"2024-04-26T20:02:03.239709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236593+0000","last_change":"2024-04-26T20:02:02.154537+0000","last_active":"2024-04-26T20:02:03.236593+0000","last_peered":"2024-04-26T20:02:03.236593+0000","last_clean":"2024-04-26T20:02:03.236593+0000","last_became_active":"2024-04-26T20:02:02.154074+0000","last_became_peered":"2024-04-26T20:02:02.154074+0000","last_unstale":"2024-04-26T20:02:03.236593+0000","last_undegraded":"2024-04-26T20:02:03.236593+0000","last_fullsized":"2024-04-26T20:02:03.236593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:59.493711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238043+0000","last_change":"2024-04-26T20:02:01.138552+0000","last_active":"2024-04-26T20:02:03.238043+0000","last_peered":"2024-04-26T20:02:03.238043+0000","last_clean":"2024-04-26T20:02:03.238043+0000","last_became_active":"2024-04-26T20:02:01.137755+0000","last_became_peered":"2024-04-26T20:02:01.137755+0000","last_unstale":"2024-04-26T20:02:03.238043+0000","last_undegraded":"2024-04-26T20:02:03.238043+0000","last_fullsized":"2024-04-26T20:02:03.238043+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:29.366170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238605+0000","last_change":"2024-04-26T20:02:02.158357+0000","last_active":"2024-04-26T20:02:03.238605+0000","last_peered":"2024-04-26T20:02:03.238605+0000","last_clean":"2024-04-26T20:02:03.238605+0000","last_became_active":"2024-04-26T20:02:02.158099+0000","last_became_peered":"2024-04-26T20:02:02.158099+0000","last_unstale":"2024-04-26T20:02:03.238605+0000","last_undegraded":"2024-04-26T20:02:03.238605+0000","last_fullsized":"2024-04-26T20:02:03.238605+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146799+0000","last_change":"2024-04-26T20:02:01.141454+0000","last_active":"2024-04-26T20:02:03.146799+0000","last_peered":"2024-04-26T20:02:03.146799+0000","last_clean":"2024-04-26T20:02:03.146799+0000","last_became_active":"2024-04-26T20:02:01.141165+0000","last_became_peered":"2024-04-26T20:02:01.141165+0000","last_unstale":"2024-04-26T20:02:03.146799+0000","last_undegraded":"2024-04-26T20:02:03.146799+0000","last_fullsized":"2024-04-26T20:02:03.146799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:54.136944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237924+0000","last_change":"2024-04-26T20:02:02.164395+0000","last_active":"2024-04-26T20:02:03.237924+0000","last_peered":"2024-04-26T20:02:03.237924+0000","last_clean":"2024-04-26T20:02:03.237924+0000","last_became_active":"2024-04-26T20:02:02.163976+0000","last_became_peered":"2024-04-26T20:02:02.163976+0000","last_unstale":"2024-04-26T20:02:03.237924+0000","last_undegraded":"2024-04-26T20:02:03.237924+0000","last_fullsized":"2024-04-26T20:02:03.237924+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236377+0000","last_change":"2024-04-26T20:02:01.149337+0000","last_active":"2024-04-26T20:02:03.236377+0000","last_peered":"2024-04-26T20:02:03.236377+0000","last_clean":"2024-04-26T20:02:03.236377+0000","last_became_active":"2024-04-26T20:02:01.147128+0000","last_became_peered":"2024-04-26T20:02:01.147128+0000","last_unstale":"2024-04-26T20:02:03.236377+0000","last_undegraded":"2024-04-26T20:02:03.236377+0000","last_fullsized":"2024-04-26T20:02:03.236377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:50.270032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238079+0000","last_change":"2024-04-26T20:02:02.156728+0000","last_active":"2024-04-26T20:02:03.238079+0000","last_peered":"2024-04-26T20:02:03.238079+0000","last_clean":"2024-04-26T20:02:03.238079+0000","last_became_active":"2024-04-26T20:02:02.156400+0000","last_became_peered":"2024-04-26T20:02:02.156400+0000","last_unstale":"2024-04-26T20:02:03.238079+0000","last_undegraded":"2024-04-26T20:02:03.238079+0000","last_fullsized":"2024-04-26T20:02:03.238079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147793+0000","last_change":"2024-04-26T20:02:01.130730+0000","last_active":"2024-04-26T20:02:03.147793+0000","last_peered":"2024-04-26T20:02:03.147793+0000","last_clean":"2024-04-26T20:02:03.147793+0000","last_became_active":"2024-04-26T20:02:01.130411+0000","last_became_peered":"2024-04-26T20:02:01.130411+0000","last_unstale":"2024-04-26T20:02:03.147793+0000","last_undegraded":"2024-04-26T20:02:03.147793+0000","last_fullsized":"2024-04-26T20:02:03.147793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:15.214957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238085+0000","last_change":"2024-04-26T20:02:02.162955+0000","last_active":"2024-04-26T20:02:03.238085+0000","last_peered":"2024-04-26T20:02:03.238085+0000","last_clean":"2024-04-26T20:02:03.238085+0000","last_became_active":"2024-04-26T20:02:02.162676+0000","last_became_peered":"2024-04-26T20:02:02.162676+0000","last_unstale":"2024-04-26T20:02:03.238085+0000","last_undegraded":"2024-04-26T20:02:03.238085+0000","last_fullsized":"2024-04-26T20:02:03.238085+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:04.136652+0000","last_change":"2024-04-26T20:02:01.145916+0000","last_active":"2024-04-26T20:02:04.136652+0000","last_peered":"2024-04-26T20:02:04.136652+0000","last_clean":"2024-04-26T20:02:04.136652+0000","last_became_active":"2024-04-26T20:02:01.145659+0000","last_became_peered":"2024-04-26T20:02:01.145659+0000","last_unstale":"2024-04-26T20:02:04.136652+0000","last_undegraded":"2024-04-26T20:02:04.136652+0000","last_fullsized":"2024-04-26T20:02:04.136652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:32.895072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146590+0000","last_change":"2024-04-26T20:02:02.148305+0000","last_active":"2024-04-26T20:02:03.146590+0000","last_peered":"2024-04-26T20:02:03.146590+0000","last_clean":"2024-04-26T20:02:03.146590+0000","last_became_active":"2024-04-26T20:02:02.148103+0000","last_became_peered":"2024-04-26T20:02:02.148103+0000","last_unstale":"2024-04-26T20:02:03.146590+0000","last_undegraded":"2024-04-26T20:02:03.146590+0000","last_fullsized":"2024-04-26T20:02:03.146590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:00.173536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151161+0000","last_change":"2024-04-26T20:02:01.143779+0000","last_active":"2024-04-26T20:02:03.151161+0000","last_peered":"2024-04-26T20:02:03.151161+0000","last_clean":"2024-04-26T20:02:03.151161+0000","last_became_active":"2024-04-26T20:02:01.143448+0000","last_became_peered":"2024-04-26T20:02:01.143448+0000","last_unstale":"2024-04-26T20:02:03.151161+0000","last_undegraded":"2024-04-26T20:02:03.151161+0000","last_fullsized":"2024-04-26T20:02:03.151161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:07:53.783522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238528+0000","last_change":"2024-04-26T20:02:02.161591+0000","last_active":"2024-04-26T20:02:03.238528+0000","last_peered":"2024-04-26T20:02:03.238528+0000","last_clean":"2024-04-26T20:02:03.238528+0000","last_became_active":"2024-04-26T20:02:02.160934+0000","last_became_peered":"2024-04-26T20:02:02.160934+0000","last_unstale":"2024-04-26T20:02:03.238528+0000","last_undegraded":"2024-04-26T20:02:03.238528+0000","last_fullsized":"2024-04-26T20:02:03.238528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237874+0000","last_change":"2024-04-26T20:02:01.145371+0000","last_active":"2024-04-26T20:02:03.237874+0000","last_peered":"2024-04-26T20:02:03.237874+0000","last_clean":"2024-04-26T20:02:03.237874+0000","last_became_active":"2024-04-26T20:02:01.145075+0000","last_became_peered":"2024-04-26T20:02:01.145075+0000","last_unstale":"2024-04-26T20:02:03.237874+0000","last_undegraded":"2024-04-26T20:02:03.237874+0000","last_fullsized":"2024-04-26T20:02:03.237874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.135333+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.135022+0000","last_became_peered":"2024-04-26T20:02:02.135022+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:17.197831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146631+0000","last_change":"2024-04-26T20:02:01.137606+0000","last_active":"2024-04-26T20:02:03.146631+0000","last_peered":"2024-04-26T20:02:03.146631+0000","last_clean":"2024-04-26T20:02:03.146631+0000","last_became_active":"2024-04-26T20:02:01.137263+0000","last_became_peered":"2024-04-26T20:02:01.137263+0000","last_unstale":"2024-04-26T20:02:03.146631+0000","last_undegraded":"2024-04-26T20:02:03.146631+0000","last_fullsized":"2024-04-26T20:02:03.146631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:43.037846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239338+0000","last_change":"2024-04-26T20:02:02.166230+0000","last_active":"2024-04-26T20:02:03.239338+0000","last_peered":"2024-04-26T20:02:03.239338+0000","last_clean":"2024-04-26T20:02:03.239338+0000","last_became_active":"2024-04-26T20:02:02.160400+0000","last_became_peered":"2024-04-26T20:02:02.160400+0000","last_unstale":"2024-04-26T20:02:03.239338+0000","last_undegraded":"2024-04-26T20:02:03.239338+0000","last_fullsized":"2024-04-26T20:02:03.239338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149809+0000","last_change":"2024-04-26T20:02:01.146319+0000","last_active":"2024-04-26T20:02:03.149809+0000","last_peered":"2024-04-26T20:02:03.149809+0000","last_clean":"2024-04-26T20:02:03.149809+0000","last_became_active":"2024-04-26T20:02:01.145982+0000","last_became_peered":"2024-04-26T20:02:01.145982+0000","last_unstale":"2024-04-26T20:02:03.149809+0000","last_undegraded":"2024-04-26T20:02:03.149809+0000","last_fullsized":"2024-04-26T20:02:03.149809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:59.976511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146311+0000","last_change":"2024-04-26T20:02:02.161688+0000","last_active":"2024-04-26T20:02:03.146311+0000","last_peered":"2024-04-26T20:02:03.146311+0000","last_clean":"2024-04-26T20:02:03.146311+0000","last_became_active":"2024-04-26T20:02:02.159715+0000","last_became_peered":"2024-04-26T20:02:02.159715+0000","last_unstale":"2024-04-26T20:02:03.146311+0000","last_undegraded":"2024-04-26T20:02:03.146311+0000","last_fullsized":"2024-04-26T20:02:03.146311+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146475+0000","last_change":"2024-04-26T20:02:01.140827+0000","last_active":"2024-04-26T20:02:03.146475+0000","last_peered":"2024-04-26T20:02:03.146475+0000","last_clean":"2024-04-26T20:02:03.146475+0000","last_became_active":"2024-04-26T20:02:01.140496+0000","last_became_peered":"2024-04-26T20:02:01.140496+0000","last_unstale":"2024-04-26T20:02:03.146475+0000","last_undegraded":"2024-04-26T20:02:03.146475+0000","last_fullsized":"2024-04-26T20:02:03.146475+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236398+0000","last_change":"2024-04-26T20:02:02.163701+0000","last_active":"2024-04-26T20:02:03.236398+0000","last_peered":"2024-04-26T20:02:03.236398+0000","last_clean":"2024-04-26T20:02:03.236398+0000","last_became_active":"2024-04-26T20:02:02.162010+0000","last_became_peered":"2024-04-26T20:02:02.162010+0000","last_unstale":"2024-04-26T20:02:03.236398+0000","last_undegraded":"2024-04-26T20:02:03.236398+0000","last_fullsized":"2024-04-26T20:02:03.236398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:36.989382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235881+0000","last_change":"2024-04-26T20:02:01.142535+0000","last_active":"2024-04-26T20:02:03.235881+0000","last_peered":"2024-04-26T20:02:03.235881+0000","last_clean":"2024-04-26T20:02:03.235881+0000","last_became_active":"2024-04-26T20:02:01.142254+0000","last_became_peered":"2024-04-26T20:02:01.142254+0000","last_unstale":"2024-04-26T20:02:03.235881+0000","last_undegraded":"2024-04-26T20:02:03.235881+0000","last_fullsized":"2024-04-26T20:02:03.235881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:17.922754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235721+0000","last_change":"2024-04-26T20:02:02.142754+0000","last_active":"2024-04-26T20:02:03.235721+0000","last_peered":"2024-04-26T20:02:03.235721+0000","last_clean":"2024-04-26T20:02:03.235721+0000","last_became_active":"2024-04-26T20:02:02.142456+0000","last_became_peered":"2024-04-26T20:02:02.142456+0000","last_unstale":"2024-04-26T20:02:03.235721+0000","last_undegraded":"2024-04-26T20:02:03.235721+0000","last_fullsized":"2024-04-26T20:02:03.235721+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:39.370350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149884+0000","last_change":"2024-04-26T20:02:01.142278+0000","last_active":"2024-04-26T20:02:03.149884+0000","last_peered":"2024-04-26T20:02:03.149884+0000","last_clean":"2024-04-26T20:02:03.149884+0000","last_became_active":"2024-04-26T20:02:01.141891+0000","last_became_peered":"2024-04-26T20:02:01.141891+0000","last_unstale":"2024-04-26T20:02:03.149884+0000","last_undegraded":"2024-04-26T20:02:03.149884+0000","last_fullsized":"2024-04-26T20:02:03.149884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:27.768532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238359+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:02:03.238359+0000","last_peered":"2024-04-26T20:02:03.238359+0000","last_clean":"2024-04-26T20:02:03.238359+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:02:03.238359+0000","last_undegraded":"2024-04-26T20:02:03.238359+0000","last_fullsized":"2024-04-26T20:02:03.238359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":855448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":264920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":265572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":266532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":265010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:09.915 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:02:09.923 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:02:11.524 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:11.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.726+0000 7f567f000640 1 Processor -- start 2024-04-26T20:02:11.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.726+0000 7f567f000640 1 -- start start 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567f000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f567806ca60 0x7f5678071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 0x7f56780b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567f000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5678073a90 con 0x7f567815f150 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567f000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f56780b1060 con 0x7f567806ca60 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f567806fee0 con 0x7f567806fa50 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567dc00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f567815f150 0x7f567806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40454/0 (socket says 172.21.15.88:40454) 2024-04-26T20:02:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.728+0000 7f567dc00640 1 -- 172.21.15.88:0/3732298276 learned_addr learned my addr 172.21.15.88:0/3732298276 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548110863 0 0) 0x7f5678073a90 con 0x7f567815f150 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f565c003520 con 0x7f567815f150 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f567d200640 1 --2- 172.21.15.88:0/3732298276 >> v2:172.21.15.132:3300/0 conn(0x7f567806ca60 0x7f5678071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f567e600640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 0x7f56780b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1348703685 0 0) 0x7f565c003520 con 0x7f567815f150 2024-04-26T20:02:11.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 msgr2=0x7f56780b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 0x7f56780b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 >> v2:172.21.15.132:3300/0 conn(0x7f567806ca60 msgr2=0x7f5678071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 --2- 172.21.15.88:0/3732298276 >> v2:172.21.15.132:3300/0 conn(0x7f567806ca60 0x7f5678071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678073d70 con 0x7f567815f150 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f5668001670 con 0x7f567815f150 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 >> v1:172.21.15.88:6789/0 conn(0x7f567815f150 legacy=0x7f567806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 0x7f565c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56780b1060 con 0x7f565c0039b0 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f565c00bad0 con 0x7f565c004800 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.729+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f565c00bcd0 con 0x7f567806fa50 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567d200640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 -- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 msgr2=0x7f565c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:11.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f567806fa50 0x7f565c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 -- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004800 msgr2=0x7f565c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:11.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f565c004800 0x7f565c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.730+0000 7f567dc00640 1 -- 172.21.15.88:0/3732298276 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c003520 con 0x7f565c0039b0 2024-04-26T20:02:11.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.731+0000 7f567dc00640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5668007fd0 tx=0x7f565c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca22f02fcf051c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:11.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.731+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668007d50 con 0x7f565c0039b0 2024-04-26T20:02:11.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.731+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668005ad0 con 0x7f565c0039b0 2024-04-26T20:02:11.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.731+0000 7f5677e00640 1 -- 172.21.15.88:0/3732298276 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668007070 con 0x7f565c0039b0 2024-04-26T20:02:11.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 -- 172.21.15.88:0/3732298276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 msgr2=0x7f565c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 --2- 172.21.15.88:0/3732298276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f565c003dd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5668007fd0 tx=0x7f565c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567e600640 1 -- 172.21.15.88:0/3732298276 reap_dead start 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 -- 172.21.15.88:0/3732298276 shutdown_connections 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 -- 172.21.15.88:0/3732298276 >> 172.21.15.88:0/3732298276 conn(0x7f56780b38a0 msgr2=0x7f56780b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 -- 172.21.15.88:0/3732298276 shutdown_connections 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.732+0000 7f567f000640 1 -- 172.21.15.88:0/3732298276 wait complete. 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.734+0000 7f567f000640 1 Processor -- start 2024-04-26T20:02:11.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.734+0000 7f567f000640 1 -- start start 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 0x7f5678141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5678150b30 0x7f56781f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f567806f250 con 0x7f565c0039b0 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f567806f5c0 con 0x7f56781f89a0 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567f000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f56780b0b20 con 0x7f5678150b30 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42920/0 (socket says 172.21.15.88:42920) 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 -- 172.21.15.88:0/66286857 learned_addr learned my addr 172.21.15.88:0/66286857 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567d200640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 0x7f5678141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 -- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5678150b30 msgr2=0x7f56781f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567e600640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5678150b30 0x7f56781f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5678150b30 0x7f56781f7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 -- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 msgr2=0x7f5678141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 0x7f5678141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567dc00640 1 -- 172.21.15.88:0/66286857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678141b50 con 0x7f565c0039b0 2024-04-26T20:02:11.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.735+0000 7f567d200640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 0x7f5678141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f567dc00640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f566800c2d0 tx=0x7f5668001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668005e40 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668041070 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680047e0 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5678153f80 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.736+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5678160620 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.737+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678069ad0 con 0x7f565c0039b0 2024-04-26T20:02:11.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.738+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f566803b070 con 0x7f565c0039b0 2024-04-26T20:02:11.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.739+0000 7f5675600640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 0x7f565806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:11.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.739+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f56680ab320 con 0x7f565c0039b0 2024-04-26T20:02:11.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.740+0000 7f567d200640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 0x7f565806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:11.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.740+0000 7f567d200640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 0x7f565806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56781450a0 tx=0x7f566400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:11.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.744+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5678141b50 con 0x7f565c0039b0 2024-04-26T20:02:11.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.940+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f567806c760 con 0x7f565806c8f0 2024-04-26T20:02:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.947+0000 7f5675600640 1 -- 172.21.15.88:0/66286857 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251843 (secure 0 0 0) 0x7f567806c760 con 0x7f565806c8f0 2024-04-26T20:02:11.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:11.954 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 msgr2=0x7f565806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 0x7f565806edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56781450a0 tx=0x7f566400b040 comp rx=0 tx=0).stop 2024-04-26T20:02:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 msgr2=0x7f5678153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f566800c2d0 tx=0x7f5668001a40 comp rx=0 tx=0).stop 2024-04-26T20:02:11.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 shutdown_connections 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f565806c8f0 0x7f565806edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5678150b30 0x7f56781f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56781f89a0 0x7f5678141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 --2- 172.21.15.88:0/66286857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f565c0039b0 0x7f5678153460 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.952+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 >> 172.21.15.88:0/66286857 conn(0x7f56780b38a0 msgr2=0x7f56780ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.953+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 shutdown_connections 2024-04-26T20:02:11.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:11.953+0000 7f567f000640 1 -- 172.21.15.88:0/66286857 wait complete. 2024-04-26T20:02:11.973 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-26T20:02:10.995138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226424,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748296,"statfs":{"total":773094113280,"available":772862255104,"internally_reserved":0,"allocated":6234112,"data_stored":3304916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.876742"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236291+0000","last_change":"2024-04-26T20:02:02.143484+0000","last_active":"2024-04-26T20:02:03.236291+0000","last_peered":"2024-04-26T20:02:03.236291+0000","last_clean":"2024-04-26T20:02:03.236291+0000","last_became_active":"2024-04-26T20:02:02.143218+0000","last_became_peered":"2024-04-26T20:02:02.143218+0000","last_unstale":"2024-04-26T20:02:03.236291+0000","last_undegraded":"2024-04-26T20:02:03.236291+0000","last_fullsized":"2024-04-26T20:02:03.236291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:10:03.607368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238903+0000","last_change":"2024-04-26T20:02:02.137481+0000","last_active":"2024-04-26T20:02:03.238903+0000","last_peered":"2024-04-26T20:02:03.238903+0000","last_clean":"2024-04-26T20:02:03.238903+0000","last_became_active":"2024-04-26T20:02:02.136583+0000","last_became_peered":"2024-04-26T20:02:02.136583+0000","last_unstale":"2024-04-26T20:02:03.238903+0000","last_undegraded":"2024-04-26T20:02:03.238903+0000","last_fullsized":"2024-04-26T20:02:03.238903+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:48.053920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239432+0000","last_change":"2024-04-26T20:02:02.168154+0000","last_active":"2024-04-26T20:02:03.239432+0000","last_peered":"2024-04-26T20:02:03.239432+0000","last_clean":"2024-04-26T20:02:03.239432+0000","last_became_active":"2024-04-26T20:02:02.167855+0000","last_became_peered":"2024-04-26T20:02:02.167855+0000","last_unstale":"2024-04-26T20:02:03.239432+0000","last_undegraded":"2024-04-26T20:02:03.239432+0000","last_fullsized":"2024-04-26T20:02:03.239432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237419+0000","last_change":"2024-04-26T20:02:02.157856+0000","last_active":"2024-04-26T20:02:03.237419+0000","last_peered":"2024-04-26T20:02:03.237419+0000","last_clean":"2024-04-26T20:02:03.237419+0000","last_became_active":"2024-04-26T20:02:02.156779+0000","last_became_peered":"2024-04-26T20:02:02.156779+0000","last_unstale":"2024-04-26T20:02:03.237419+0000","last_undegraded":"2024-04-26T20:02:03.237419+0000","last_fullsized":"2024-04-26T20:02:03.237419+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:26.406175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236036+0000","last_change":"2024-04-26T20:02:02.164743+0000","last_active":"2024-04-26T20:02:03.236036+0000","last_peered":"2024-04-26T20:02:03.236036+0000","last_clean":"2024-04-26T20:02:03.236036+0000","last_became_active":"2024-04-26T20:02:02.164403+0000","last_became_peered":"2024-04-26T20:02:02.164403+0000","last_unstale":"2024-04-26T20:02:03.236036+0000","last_undegraded":"2024-04-26T20:02:03.236036+0000","last_fullsized":"2024-04-26T20:02:03.236036+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:10.126360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238106+0000","last_change":"2024-04-26T20:02:02.153689+0000","last_active":"2024-04-26T20:02:03.238106+0000","last_peered":"2024-04-26T20:02:03.238106+0000","last_clean":"2024-04-26T20:02:03.238106+0000","last_became_active":"2024-04-26T20:02:02.153476+0000","last_became_peered":"2024-04-26T20:02:02.153476+0000","last_unstale":"2024-04-26T20:02:03.238106+0000","last_undegraded":"2024-04-26T20:02:03.238106+0000","last_fullsized":"2024-04-26T20:02:03.238106+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151473+0000","last_change":"2024-04-26T20:02:02.163749+0000","last_active":"2024-04-26T20:02:03.151473+0000","last_peered":"2024-04-26T20:02:03.151473+0000","last_clean":"2024-04-26T20:02:03.151473+0000","last_became_active":"2024-04-26T20:02:02.162498+0000","last_became_peered":"2024-04-26T20:02:02.162498+0000","last_unstale":"2024-04-26T20:02:03.151473+0000","last_undegraded":"2024-04-26T20:02:03.151473+0000","last_fullsized":"2024-04-26T20:02:03.151473+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235764+0000","last_change":"2024-04-26T20:02:02.155433+0000","last_active":"2024-04-26T20:02:03.235764+0000","last_peered":"2024-04-26T20:02:03.235764+0000","last_clean":"2024-04-26T20:02:03.235764+0000","last_became_active":"2024-04-26T20:02:02.155167+0000","last_became_peered":"2024-04-26T20:02:02.155167+0000","last_unstale":"2024-04-26T20:02:03.235764+0000","last_undegraded":"2024-04-26T20:02:03.235764+0000","last_fullsized":"2024-04-26T20:02:03.235764+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.488190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149596+0000","last_change":"2024-04-26T20:02:02.145050+0000","last_active":"2024-04-26T20:02:03.149596+0000","last_peered":"2024-04-26T20:02:03.149596+0000","last_clean":"2024-04-26T20:02:03.149596+0000","last_became_active":"2024-04-26T20:02:02.144748+0000","last_became_peered":"2024-04-26T20:02:02.144748+0000","last_unstale":"2024-04-26T20:02:03.149596+0000","last_undegraded":"2024-04-26T20:02:03.149596+0000","last_fullsized":"2024-04-26T20:02:03.149596+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:27.608630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147062+0000","last_change":"2024-04-26T20:02:02.157240+0000","last_active":"2024-04-26T20:02:03.147062+0000","last_peered":"2024-04-26T20:02:03.147062+0000","last_clean":"2024-04-26T20:02:03.147062+0000","last_became_active":"2024-04-26T20:02:02.156963+0000","last_became_peered":"2024-04-26T20:02:02.156963+0000","last_unstale":"2024-04-26T20:02:03.147062+0000","last_undegraded":"2024-04-26T20:02:03.147062+0000","last_fullsized":"2024-04-26T20:02:03.147062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:01.598249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147030+0000","last_change":"2024-04-26T20:02:02.149796+0000","last_active":"2024-04-26T20:02:03.147030+0000","last_peered":"2024-04-26T20:02:03.147030+0000","last_clean":"2024-04-26T20:02:03.147030+0000","last_became_active":"2024-04-26T20:02:02.146559+0000","last_became_peered":"2024-04-26T20:02:02.146559+0000","last_unstale":"2024-04-26T20:02:03.147030+0000","last_undegraded":"2024-04-26T20:02:03.147030+0000","last_fullsized":"2024-04-26T20:02:03.147030+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:23.159614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237551+0000","last_change":"2024-04-26T20:02:02.163072+0000","last_active":"2024-04-26T20:02:03.237551+0000","last_peered":"2024-04-26T20:02:03.237551+0000","last_clean":"2024-04-26T20:02:03.237551+0000","last_became_active":"2024-04-26T20:02:02.162527+0000","last_became_peered":"2024-04-26T20:02:02.162527+0000","last_unstale":"2024-04-26T20:02:03.237551+0000","last_undegraded":"2024-04-26T20:02:03.237551+0000","last_fullsized":"2024-04-26T20:02:03.237551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:39.029864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151035+0000","last_change":"2024-04-26T20:02:02.165794+0000","last_active":"2024-04-26T20:02:03.151035+0000","last_peered":"2024-04-26T20:02:03.151035+0000","last_clean":"2024-04-26T20:02:03.151035+0000","last_became_active":"2024-04-26T20:02:02.165488+0000","last_became_peered":"2024-04-26T20:02:02.165488+0000","last_unstale":"2024-04-26T20:02:03.151035+0000","last_undegraded":"2024-04-26T20:02:03.151035+0000","last_fullsized":"2024-04-26T20:02:03.151035+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.541941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149361+0000","last_change":"2024-04-26T20:02:02.144523+0000","last_active":"2024-04-26T20:02:03.149361+0000","last_peered":"2024-04-26T20:02:03.149361+0000","last_clean":"2024-04-26T20:02:03.149361+0000","last_became_active":"2024-04-26T20:02:02.143456+0000","last_became_peered":"2024-04-26T20:02:02.143456+0000","last_unstale":"2024-04-26T20:02:03.149361+0000","last_undegraded":"2024-04-26T20:02:03.149361+0000","last_fullsized":"2024-04-26T20:02:03.149361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:41.021198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.152033+0000","last_change":"2024-04-26T20:02:02.141347+0000","last_active":"2024-04-26T20:02:03.152033+0000","last_peered":"2024-04-26T20:02:03.152033+0000","last_clean":"2024-04-26T20:02:03.152033+0000","last_became_active":"2024-04-26T20:02:02.141029+0000","last_became_peered":"2024-04-26T20:02:02.141029+0000","last_unstale":"2024-04-26T20:02:03.152033+0000","last_undegraded":"2024-04-26T20:02:03.152033+0000","last_fullsized":"2024-04-26T20:02:03.152033+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238361+0000","last_change":"2024-04-26T20:02:02.156328+0000","last_active":"2024-04-26T20:02:03.238361+0000","last_peered":"2024-04-26T20:02:03.238361+0000","last_clean":"2024-04-26T20:02:03.238361+0000","last_became_active":"2024-04-26T20:02:02.155610+0000","last_became_peered":"2024-04-26T20:02:02.155610+0000","last_unstale":"2024-04-26T20:02:03.238361+0000","last_undegraded":"2024-04-26T20:02:03.238361+0000","last_fullsized":"2024-04-26T20:02:03.238361+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239089+0000","last_change":"2024-04-26T20:02:02.163223+0000","last_active":"2024-04-26T20:02:03.239089+0000","last_peered":"2024-04-26T20:02:03.239089+0000","last_clean":"2024-04-26T20:02:03.239089+0000","last_became_active":"2024-04-26T20:02:02.160865+0000","last_became_peered":"2024-04-26T20:02:02.160865+0000","last_unstale":"2024-04-26T20:02:03.239089+0000","last_undegraded":"2024-04-26T20:02:03.239089+0000","last_fullsized":"2024-04-26T20:02:03.239089+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:33.436022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239288+0000","last_change":"2024-04-26T20:02:02.167054+0000","last_active":"2024-04-26T20:02:03.239288+0000","last_peered":"2024-04-26T20:02:03.239288+0000","last_clean":"2024-04-26T20:02:03.239288+0000","last_became_active":"2024-04-26T20:02:02.166210+0000","last_became_peered":"2024-04-26T20:02:02.166210+0000","last_unstale":"2024-04-26T20:02:03.239288+0000","last_undegraded":"2024-04-26T20:02:03.239288+0000","last_fullsized":"2024-04-26T20:02:03.239288+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:55.792059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236507+0000","last_change":"2024-04-26T20:02:02.140498+0000","last_active":"2024-04-26T20:02:03.236507+0000","last_peered":"2024-04-26T20:02:03.236507+0000","last_clean":"2024-04-26T20:02:03.236507+0000","last_became_active":"2024-04-26T20:02:02.140222+0000","last_became_peered":"2024-04-26T20:02:02.140222+0000","last_unstale":"2024-04-26T20:02:03.236507+0000","last_undegraded":"2024-04-26T20:02:03.236507+0000","last_fullsized":"2024-04-26T20:02:03.236507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:49.790277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237151+0000","last_change":"2024-04-26T20:02:02.141213+0000","last_active":"2024-04-26T20:02:03.237151+0000","last_peered":"2024-04-26T20:02:03.237151+0000","last_clean":"2024-04-26T20:02:03.237151+0000","last_became_active":"2024-04-26T20:02:02.140950+0000","last_became_peered":"2024-04-26T20:02:02.140950+0000","last_unstale":"2024-04-26T20:02:03.237151+0000","last_undegraded":"2024-04-26T20:02:03.237151+0000","last_fullsized":"2024-04-26T20:02:03.237151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:10.121113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238997+0000","last_change":"2024-04-26T20:02:02.165402+0000","last_active":"2024-04-26T20:02:03.238997+0000","last_peered":"2024-04-26T20:02:03.238997+0000","last_clean":"2024-04-26T20:02:03.238997+0000","last_became_active":"2024-04-26T20:02:02.165039+0000","last_became_peered":"2024-04-26T20:02:02.165039+0000","last_unstale":"2024-04-26T20:02:03.238997+0000","last_undegraded":"2024-04-26T20:02:03.238997+0000","last_fullsized":"2024-04-26T20:02:03.238997+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235925+0000","last_change":"2024-04-26T20:02:02.146950+0000","last_active":"2024-04-26T20:02:03.235925+0000","last_peered":"2024-04-26T20:02:03.235925+0000","last_clean":"2024-04-26T20:02:03.235925+0000","last_became_active":"2024-04-26T20:02:02.146326+0000","last_became_peered":"2024-04-26T20:02:02.146326+0000","last_unstale":"2024-04-26T20:02:03.235925+0000","last_undegraded":"2024-04-26T20:02:03.235925+0000","last_fullsized":"2024-04-26T20:02:03.235925+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:43.597063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239874+0000","last_change":"2024-04-26T20:02:02.157997+0000","last_active":"2024-04-26T20:02:03.239874+0000","last_peered":"2024-04-26T20:02:03.239874+0000","last_clean":"2024-04-26T20:02:03.239874+0000","last_became_active":"2024-04-26T20:02:02.155921+0000","last_became_peered":"2024-04-26T20:02:02.155921+0000","last_unstale":"2024-04-26T20:02:03.239874+0000","last_undegraded":"2024-04-26T20:02:03.239874+0000","last_fullsized":"2024-04-26T20:02:03.239874+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:20.786310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149079+0000","last_change":"2024-04-26T20:02:02.160986+0000","last_active":"2024-04-26T20:02:03.149079+0000","last_peered":"2024-04-26T20:02:03.149079+0000","last_clean":"2024-04-26T20:02:03.149079+0000","last_became_active":"2024-04-26T20:02:02.160693+0000","last_became_peered":"2024-04-26T20:02:02.160693+0000","last_unstale":"2024-04-26T20:02:03.149079+0000","last_undegraded":"2024-04-26T20:02:03.149079+0000","last_fullsized":"2024-04-26T20:02:03.149079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:42.992497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236630+0000","last_change":"2024-04-26T20:02:02.149529+0000","last_active":"2024-04-26T20:02:03.236630+0000","last_peered":"2024-04-26T20:02:03.236630+0000","last_clean":"2024-04-26T20:02:03.236630+0000","last_became_active":"2024-04-26T20:02:02.148380+0000","last_became_peered":"2024-04-26T20:02:02.148380+0000","last_unstale":"2024-04-26T20:02:03.236630+0000","last_undegraded":"2024-04-26T20:02:03.236630+0000","last_fullsized":"2024-04-26T20:02:03.236630+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:41.211746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237736+0000","last_change":"2024-04-26T20:02:02.157317+0000","last_active":"2024-04-26T20:02:03.237736+0000","last_peered":"2024-04-26T20:02:03.237736+0000","last_clean":"2024-04-26T20:02:03.237736+0000","last_became_active":"2024-04-26T20:02:02.157063+0000","last_became_peered":"2024-04-26T20:02:02.157063+0000","last_unstale":"2024-04-26T20:02:03.237736+0000","last_undegraded":"2024-04-26T20:02:03.237736+0000","last_fullsized":"2024-04-26T20:02:03.237736+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:18.938594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237967+0000","last_change":"2024-04-26T20:02:02.166022+0000","last_active":"2024-04-26T20:02:03.237967+0000","last_peered":"2024-04-26T20:02:03.237967+0000","last_clean":"2024-04-26T20:02:03.237967+0000","last_became_active":"2024-04-26T20:02:02.165717+0000","last_became_peered":"2024-04-26T20:02:02.165717+0000","last_unstale":"2024-04-26T20:02:03.237967+0000","last_undegraded":"2024-04-26T20:02:03.237967+0000","last_fullsized":"2024-04-26T20:02:03.237967+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236343+0000","last_change":"2024-04-26T20:02:02.159795+0000","last_active":"2024-04-26T20:02:03.236343+0000","last_peered":"2024-04-26T20:02:03.236343+0000","last_clean":"2024-04-26T20:02:03.236343+0000","last_became_active":"2024-04-26T20:02:02.159453+0000","last_became_peered":"2024-04-26T20:02:02.159453+0000","last_unstale":"2024-04-26T20:02:03.236343+0000","last_undegraded":"2024-04-26T20:02:03.236343+0000","last_fullsized":"2024-04-26T20:02:03.236343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:13.456162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237886+0000","last_change":"2024-04-26T20:02:02.164919+0000","last_active":"2024-04-26T20:02:03.237886+0000","last_peered":"2024-04-26T20:02:03.237886+0000","last_clean":"2024-04-26T20:02:03.237886+0000","last_became_active":"2024-04-26T20:02:02.164660+0000","last_became_peered":"2024-04-26T20:02:02.164660+0000","last_unstale":"2024-04-26T20:02:03.237886+0000","last_undegraded":"2024-04-26T20:02:03.237886+0000","last_fullsized":"2024-04-26T20:02:03.237886+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:25:34.099908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238768+0000","last_change":"2024-04-26T20:02:02.165762+0000","last_active":"2024-04-26T20:02:03.238768+0000","last_peered":"2024-04-26T20:02:03.238768+0000","last_clean":"2024-04-26T20:02:03.238768+0000","last_became_active":"2024-04-26T20:02:02.165478+0000","last_became_peered":"2024-04-26T20:02:02.165478+0000","last_unstale":"2024-04-26T20:02:03.238768+0000","last_undegraded":"2024-04-26T20:02:03.238768+0000","last_fullsized":"2024-04-26T20:02:03.238768+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235608+0000","last_change":"2024-04-26T20:02:02.143180+0000","last_active":"2024-04-26T20:02:03.235608+0000","last_peered":"2024-04-26T20:02:03.235608+0000","last_clean":"2024-04-26T20:02:03.235608+0000","last_became_active":"2024-04-26T20:02:02.138046+0000","last_became_peered":"2024-04-26T20:02:02.138046+0000","last_unstale":"2024-04-26T20:02:03.235608+0000","last_undegraded":"2024-04-26T20:02:03.235608+0000","last_fullsized":"2024-04-26T20:02:03.235608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:01:53.041114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237793+0000","last_change":"2024-04-26T20:02:02.149273+0000","last_active":"2024-04-26T20:02:03.237793+0000","last_peered":"2024-04-26T20:02:03.237793+0000","last_clean":"2024-04-26T20:02:03.237793+0000","last_became_active":"2024-04-26T20:02:02.148981+0000","last_became_peered":"2024-04-26T20:02:02.148981+0000","last_unstale":"2024-04-26T20:02:03.237793+0000","last_undegraded":"2024-04-26T20:02:03.237793+0000","last_fullsized":"2024-04-26T20:02:03.237793+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238974+0000","last_change":"2024-04-26T20:02:02.163204+0000","last_active":"2024-04-26T20:02:03.238974+0000","last_peered":"2024-04-26T20:02:03.238974+0000","last_clean":"2024-04-26T20:02:03.238974+0000","last_became_active":"2024-04-26T20:02:02.162514+0000","last_became_peered":"2024-04-26T20:02:02.162514+0000","last_unstale":"2024-04-26T20:02:03.238974+0000","last_undegraded":"2024-04-26T20:02:03.238974+0000","last_fullsized":"2024-04-26T20:02:03.238974+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239386+0000","last_change":"2024-04-26T20:02:01.144979+0000","last_active":"2024-04-26T20:02:03.239386+0000","last_peered":"2024-04-26T20:02:03.239386+0000","last_clean":"2024-04-26T20:02:03.239386+0000","last_became_active":"2024-04-26T20:02:01.144650+0000","last_became_peered":"2024-04-26T20:02:01.144650+0000","last_unstale":"2024-04-26T20:02:03.239386+0000","last_undegraded":"2024-04-26T20:02:03.239386+0000","last_fullsized":"2024-04-26T20:02:03.239386+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147523+0000","last_change":"2024-04-26T20:02:02.155432+0000","last_active":"2024-04-26T20:02:03.147523+0000","last_peered":"2024-04-26T20:02:03.147523+0000","last_clean":"2024-04-26T20:02:03.147523+0000","last_became_active":"2024-04-26T20:02:02.153246+0000","last_became_peered":"2024-04-26T20:02:02.153246+0000","last_unstale":"2024-04-26T20:02:03.147523+0000","last_undegraded":"2024-04-26T20:02:03.147523+0000","last_fullsized":"2024-04-26T20:02:03.147523+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:31.583720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239186+0000","last_change":"2024-04-26T20:02:01.149903+0000","last_active":"2024-04-26T20:02:03.239186+0000","last_peered":"2024-04-26T20:02:03.239186+0000","last_clean":"2024-04-26T20:02:03.239186+0000","last_became_active":"2024-04-26T20:02:01.149641+0000","last_became_peered":"2024-04-26T20:02:01.149641+0000","last_unstale":"2024-04-26T20:02:03.239186+0000","last_undegraded":"2024-04-26T20:02:03.239186+0000","last_fullsized":"2024-04-26T20:02:03.239186+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:25.390334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238151+0000","last_change":"2024-04-26T20:02:02.149195+0000","last_active":"2024-04-26T20:02:03.238151+0000","last_peered":"2024-04-26T20:02:03.238151+0000","last_clean":"2024-04-26T20:02:03.238151+0000","last_became_active":"2024-04-26T20:02:02.148929+0000","last_became_peered":"2024-04-26T20:02:02.148929+0000","last_unstale":"2024-04-26T20:02:03.238151+0000","last_undegraded":"2024-04-26T20:02:03.238151+0000","last_fullsized":"2024-04-26T20:02:03.238151+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238738+0000","last_change":"2024-04-26T20:02:01.148651+0000","last_active":"2024-04-26T20:02:03.238738+0000","last_peered":"2024-04-26T20:02:03.238738+0000","last_clean":"2024-04-26T20:02:03.238738+0000","last_became_active":"2024-04-26T20:02:01.148343+0000","last_became_peered":"2024-04-26T20:02:01.148343+0000","last_unstale":"2024-04-26T20:02:03.238738+0000","last_undegraded":"2024-04-26T20:02:03.238738+0000","last_fullsized":"2024-04-26T20:02:03.238738+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238255+0000","last_change":"2024-04-26T20:02:02.161891+0000","last_active":"2024-04-26T20:02:03.238255+0000","last_peered":"2024-04-26T20:02:03.238255+0000","last_clean":"2024-04-26T20:02:03.238255+0000","last_became_active":"2024-04-26T20:02:02.161338+0000","last_became_peered":"2024-04-26T20:02:02.161338+0000","last_unstale":"2024-04-26T20:02:03.238255+0000","last_undegraded":"2024-04-26T20:02:03.238255+0000","last_fullsized":"2024-04-26T20:02:03.238255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:30.277326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237809+0000","last_change":"2024-04-26T20:02:01.148830+0000","last_active":"2024-04-26T20:02:03.237809+0000","last_peered":"2024-04-26T20:02:03.237809+0000","last_clean":"2024-04-26T20:02:03.237809+0000","last_became_active":"2024-04-26T20:02:01.148527+0000","last_became_peered":"2024-04-26T20:02:01.148527+0000","last_unstale":"2024-04-26T20:02:03.237809+0000","last_undegraded":"2024-04-26T20:02:03.237809+0000","last_fullsized":"2024-04-26T20:02:03.237809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:49:40.005357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236945+0000","last_change":"2024-04-26T20:02:02.147161+0000","last_active":"2024-04-26T20:02:03.236945+0000","last_peered":"2024-04-26T20:02:03.236945+0000","last_clean":"2024-04-26T20:02:03.236945+0000","last_became_active":"2024-04-26T20:02:02.146700+0000","last_became_peered":"2024-04-26T20:02:02.146700+0000","last_unstale":"2024-04-26T20:02:03.236945+0000","last_undegraded":"2024-04-26T20:02:03.236945+0000","last_fullsized":"2024-04-26T20:02:03.236945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.083746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149173+0000","last_change":"2024-04-26T20:02:01.147048+0000","last_active":"2024-04-26T20:02:03.149173+0000","last_peered":"2024-04-26T20:02:03.149173+0000","last_clean":"2024-04-26T20:02:03.149173+0000","last_became_active":"2024-04-26T20:02:01.146474+0000","last_became_peered":"2024-04-26T20:02:01.146474+0000","last_unstale":"2024-04-26T20:02:03.149173+0000","last_undegraded":"2024-04-26T20:02:03.149173+0000","last_fullsized":"2024-04-26T20:02:03.149173+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:01.504200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237474+0000","last_change":"2024-04-26T20:02:02.143048+0000","last_active":"2024-04-26T20:02:03.237474+0000","last_peered":"2024-04-26T20:02:03.237474+0000","last_clean":"2024-04-26T20:02:03.237474+0000","last_became_active":"2024-04-26T20:02:02.142774+0000","last_became_peered":"2024-04-26T20:02:02.142774+0000","last_unstale":"2024-04-26T20:02:03.237474+0000","last_undegraded":"2024-04-26T20:02:03.237474+0000","last_fullsized":"2024-04-26T20:02:03.237474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:15.594485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238274+0000","last_change":"2024-04-26T20:02:01.147107+0000","last_active":"2024-04-26T20:02:03.238274+0000","last_peered":"2024-04-26T20:02:03.238274+0000","last_clean":"2024-04-26T20:02:03.238274+0000","last_became_active":"2024-04-26T20:02:01.146850+0000","last_became_peered":"2024-04-26T20:02:01.146850+0000","last_unstale":"2024-04-26T20:02:03.238274+0000","last_undegraded":"2024-04-26T20:02:03.238274+0000","last_fullsized":"2024-04-26T20:02:03.238274+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:01:59.157695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149081+0000","last_change":"2024-04-26T20:02:02.166936+0000","last_active":"2024-04-26T20:02:03.149081+0000","last_peered":"2024-04-26T20:02:03.149081+0000","last_clean":"2024-04-26T20:02:03.149081+0000","last_became_active":"2024-04-26T20:02:02.166612+0000","last_became_peered":"2024-04-26T20:02:02.166612+0000","last_unstale":"2024-04-26T20:02:03.149081+0000","last_undegraded":"2024-04-26T20:02:03.149081+0000","last_fullsized":"2024-04-26T20:02:03.149081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:09:54.604807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236393+0000","last_change":"2024-04-26T20:02:01.145719+0000","last_active":"2024-04-26T20:02:03.236393+0000","last_peered":"2024-04-26T20:02:03.236393+0000","last_clean":"2024-04-26T20:02:03.236393+0000","last_became_active":"2024-04-26T20:02:01.145254+0000","last_became_peered":"2024-04-26T20:02:01.145254+0000","last_unstale":"2024-04-26T20:02:03.236393+0000","last_undegraded":"2024-04-26T20:02:03.236393+0000","last_fullsized":"2024-04-26T20:02:03.236393+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236081+0000","last_change":"2024-04-26T20:02:02.162161+0000","last_active":"2024-04-26T20:02:03.236081+0000","last_peered":"2024-04-26T20:02:03.236081+0000","last_clean":"2024-04-26T20:02:03.236081+0000","last_became_active":"2024-04-26T20:02:02.159842+0000","last_became_peered":"2024-04-26T20:02:02.159842+0000","last_unstale":"2024-04-26T20:02:03.236081+0000","last_undegraded":"2024-04-26T20:02:03.236081+0000","last_fullsized":"2024-04-26T20:02:03.236081+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:01.432928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149634+0000","last_change":"2024-04-26T20:02:01.140193+0000","last_active":"2024-04-26T20:02:03.149634+0000","last_peered":"2024-04-26T20:02:03.149634+0000","last_clean":"2024-04-26T20:02:03.149634+0000","last_became_active":"2024-04-26T20:02:01.139881+0000","last_became_peered":"2024-04-26T20:02:01.139881+0000","last_unstale":"2024-04-26T20:02:03.149634+0000","last_undegraded":"2024-04-26T20:02:03.149634+0000","last_fullsized":"2024-04-26T20:02:03.149634+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:24.798986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150274+0000","last_change":"2024-04-26T20:02:02.142384+0000","last_active":"2024-04-26T20:02:03.150274+0000","last_peered":"2024-04-26T20:02:03.150274+0000","last_clean":"2024-04-26T20:02:03.150274+0000","last_became_active":"2024-04-26T20:02:02.142101+0000","last_became_peered":"2024-04-26T20:02:02.142101+0000","last_unstale":"2024-04-26T20:02:03.150274+0000","last_undegraded":"2024-04-26T20:02:03.150274+0000","last_fullsized":"2024-04-26T20:02:03.150274+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:53.380303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240368+0000","last_change":"2024-04-26T20:02:01.157641+0000","last_active":"2024-04-26T20:02:03.240368+0000","last_peered":"2024-04-26T20:02:03.240368+0000","last_clean":"2024-04-26T20:02:03.240368+0000","last_became_active":"2024-04-26T20:02:01.157289+0000","last_became_peered":"2024-04-26T20:02:01.157289+0000","last_unstale":"2024-04-26T20:02:03.240368+0000","last_undegraded":"2024-04-26T20:02:03.240368+0000","last_fullsized":"2024-04-26T20:02:03.240368+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.240208+0000","last_change":"2024-04-26T20:02:02.162233+0000","last_active":"2024-04-26T20:02:03.240208+0000","last_peered":"2024-04-26T20:02:03.240208+0000","last_clean":"2024-04-26T20:02:03.240208+0000","last_became_active":"2024-04-26T20:02:02.161524+0000","last_became_peered":"2024-04-26T20:02:02.161524+0000","last_unstale":"2024-04-26T20:02:03.240208+0000","last_undegraded":"2024-04-26T20:02:03.240208+0000","last_fullsized":"2024-04-26T20:02:03.240208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:12.313138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239429+0000","last_change":"2024-04-26T20:02:01.143055+0000","last_active":"2024-04-26T20:02:03.239429+0000","last_peered":"2024-04-26T20:02:03.239429+0000","last_clean":"2024-04-26T20:02:03.239429+0000","last_became_active":"2024-04-26T20:02:01.142764+0000","last_became_peered":"2024-04-26T20:02:01.142764+0000","last_unstale":"2024-04-26T20:02:03.239429+0000","last_undegraded":"2024-04-26T20:02:03.239429+0000","last_fullsized":"2024-04-26T20:02:03.239429+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:38.014023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146434+0000","last_change":"2024-04-26T20:02:02.153895+0000","last_active":"2024-04-26T20:02:03.146434+0000","last_peered":"2024-04-26T20:02:03.146434+0000","last_clean":"2024-04-26T20:02:03.146434+0000","last_became_active":"2024-04-26T20:02:02.153646+0000","last_became_peered":"2024-04-26T20:02:02.153646+0000","last_unstale":"2024-04-26T20:02:03.146434+0000","last_undegraded":"2024-04-26T20:02:03.146434+0000","last_fullsized":"2024-04-26T20:02:03.146434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:33.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239961+0000","last_change":"2024-04-26T20:02:01.157279+0000","last_active":"2024-04-26T20:02:03.239961+0000","last_peered":"2024-04-26T20:02:03.239961+0000","last_clean":"2024-04-26T20:02:03.239961+0000","last_became_active":"2024-04-26T20:02:01.156862+0000","last_became_peered":"2024-04-26T20:02:01.156862+0000","last_unstale":"2024-04-26T20:02:03.239961+0000","last_undegraded":"2024-04-26T20:02:03.239961+0000","last_fullsized":"2024-04-26T20:02:03.239961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239540+0000","last_change":"2024-04-26T20:02:02.164666+0000","last_active":"2024-04-26T20:02:03.239540+0000","last_peered":"2024-04-26T20:02:03.239540+0000","last_clean":"2024-04-26T20:02:03.239540+0000","last_became_active":"2024-04-26T20:02:02.163858+0000","last_became_peered":"2024-04-26T20:02:02.163858+0000","last_unstale":"2024-04-26T20:02:03.239540+0000","last_undegraded":"2024-04-26T20:02:03.239540+0000","last_fullsized":"2024-04-26T20:02:03.239540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:57.215063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150354+0000","last_change":"2024-04-26T20:02:01.140072+0000","last_active":"2024-04-26T20:02:03.150354+0000","last_peered":"2024-04-26T20:02:03.150354+0000","last_clean":"2024-04-26T20:02:03.150354+0000","last_became_active":"2024-04-26T20:02:01.139748+0000","last_became_peered":"2024-04-26T20:02:01.139748+0000","last_unstale":"2024-04-26T20:02:03.150354+0000","last_undegraded":"2024-04-26T20:02:03.150354+0000","last_fullsized":"2024-04-26T20:02:03.150354+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236560+0000","last_change":"2024-04-26T20:02:02.162563+0000","last_active":"2024-04-26T20:02:03.236560+0000","last_peered":"2024-04-26T20:02:03.236560+0000","last_clean":"2024-04-26T20:02:03.236560+0000","last_became_active":"2024-04-26T20:02:02.161077+0000","last_became_peered":"2024-04-26T20:02:02.161077+0000","last_unstale":"2024-04-26T20:02:03.236560+0000","last_undegraded":"2024-04-26T20:02:03.236560+0000","last_fullsized":"2024-04-26T20:02:03.236560+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:57.134404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237650+0000","last_change":"2024-04-26T20:02:01.138772+0000","last_active":"2024-04-26T20:02:03.237650+0000","last_peered":"2024-04-26T20:02:03.237650+0000","last_clean":"2024-04-26T20:02:03.237650+0000","last_became_active":"2024-04-26T20:02:01.138304+0000","last_became_peered":"2024-04-26T20:02:01.138304+0000","last_unstale":"2024-04-26T20:02:03.237650+0000","last_undegraded":"2024-04-26T20:02:03.237650+0000","last_fullsized":"2024-04-26T20:02:03.237650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:31.876037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238949+0000","last_change":"2024-04-26T20:02:02.167078+0000","last_active":"2024-04-26T20:02:03.238949+0000","last_peered":"2024-04-26T20:02:03.238949+0000","last_clean":"2024-04-26T20:02:03.238949+0000","last_became_active":"2024-04-26T20:02:02.166744+0000","last_became_peered":"2024-04-26T20:02:02.166744+0000","last_unstale":"2024-04-26T20:02:03.238949+0000","last_undegraded":"2024-04-26T20:02:03.238949+0000","last_fullsized":"2024-04-26T20:02:03.238949+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150446+0000","last_change":"2024-04-26T20:02:01.146462+0000","last_active":"2024-04-26T20:02:03.150446+0000","last_peered":"2024-04-26T20:02:03.150446+0000","last_clean":"2024-04-26T20:02:03.150446+0000","last_became_active":"2024-04-26T20:02:01.144773+0000","last_became_peered":"2024-04-26T20:02:01.144773+0000","last_unstale":"2024-04-26T20:02:03.150446+0000","last_undegraded":"2024-04-26T20:02:03.150446+0000","last_fullsized":"2024-04-26T20:02:03.150446+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:31.491400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.150047+0000","last_change":"2024-04-26T20:02:02.156782+0000","last_active":"2024-04-26T20:02:03.150047+0000","last_peered":"2024-04-26T20:02:03.150047+0000","last_clean":"2024-04-26T20:02:03.150047+0000","last_became_active":"2024-04-26T20:02:02.156432+0000","last_became_peered":"2024-04-26T20:02:02.156432+0000","last_unstale":"2024-04-26T20:02:03.150047+0000","last_undegraded":"2024-04-26T20:02:03.150047+0000","last_fullsized":"2024-04-26T20:02:03.150047+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:53.796204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236547+0000","last_change":"2024-04-26T20:02:01.151256+0000","last_active":"2024-04-26T20:02:03.236547+0000","last_peered":"2024-04-26T20:02:03.236547+0000","last_clean":"2024-04-26T20:02:03.236547+0000","last_became_active":"2024-04-26T20:02:01.150923+0000","last_became_peered":"2024-04-26T20:02:01.150923+0000","last_unstale":"2024-04-26T20:02:03.236547+0000","last_undegraded":"2024-04-26T20:02:03.236547+0000","last_fullsized":"2024-04-26T20:02:03.236547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:27:52.364462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239686+0000","last_change":"2024-04-26T20:02:02.157104+0000","last_active":"2024-04-26T20:02:03.239686+0000","last_peered":"2024-04-26T20:02:03.239686+0000","last_clean":"2024-04-26T20:02:03.239686+0000","last_became_active":"2024-04-26T20:02:02.156832+0000","last_became_peered":"2024-04-26T20:02:02.156832+0000","last_unstale":"2024-04-26T20:02:03.239686+0000","last_undegraded":"2024-04-26T20:02:03.239686+0000","last_fullsized":"2024-04-26T20:02:03.239686+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238418+0000","last_change":"2024-04-26T20:02:01.153327+0000","last_active":"2024-04-26T20:02:03.238418+0000","last_peered":"2024-04-26T20:02:03.238418+0000","last_clean":"2024-04-26T20:02:03.238418+0000","last_became_active":"2024-04-26T20:02:01.152975+0000","last_became_peered":"2024-04-26T20:02:01.152975+0000","last_unstale":"2024-04-26T20:02:03.238418+0000","last_undegraded":"2024-04-26T20:02:03.238418+0000","last_fullsized":"2024-04-26T20:02:03.238418+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239726+0000","last_change":"2024-04-26T20:02:02.168360+0000","last_active":"2024-04-26T20:02:03.239726+0000","last_peered":"2024-04-26T20:02:03.239726+0000","last_clean":"2024-04-26T20:02:03.239726+0000","last_became_active":"2024-04-26T20:02:02.166272+0000","last_became_peered":"2024-04-26T20:02:02.166272+0000","last_unstale":"2024-04-26T20:02:03.239726+0000","last_undegraded":"2024-04-26T20:02:03.239726+0000","last_fullsized":"2024-04-26T20:02:03.239726+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239268+0000","last_change":"2024-04-26T20:02:01.142280+0000","last_active":"2024-04-26T20:02:03.239268+0000","last_peered":"2024-04-26T20:02:03.239268+0000","last_clean":"2024-04-26T20:02:03.239268+0000","last_became_active":"2024-04-26T20:02:01.142004+0000","last_became_peered":"2024-04-26T20:02:01.142004+0000","last_unstale":"2024-04-26T20:02:03.239268+0000","last_undegraded":"2024-04-26T20:02:03.239268+0000","last_fullsized":"2024-04-26T20:02:03.239268+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:28.587601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239134+0000","last_change":"2024-04-26T20:02:02.158922+0000","last_active":"2024-04-26T20:02:03.239134+0000","last_peered":"2024-04-26T20:02:03.239134+0000","last_clean":"2024-04-26T20:02:03.239134+0000","last_became_active":"2024-04-26T20:02:02.158537+0000","last_became_peered":"2024-04-26T20:02:02.158537+0000","last_unstale":"2024-04-26T20:02:03.239134+0000","last_undegraded":"2024-04-26T20:02:03.239134+0000","last_fullsized":"2024-04-26T20:02:03.239134+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:30.382011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239067+0000","last_change":"2024-04-26T20:02:01.140742+0000","last_active":"2024-04-26T20:02:03.239067+0000","last_peered":"2024-04-26T20:02:03.239067+0000","last_clean":"2024-04-26T20:02:03.239067+0000","last_became_active":"2024-04-26T20:02:01.140411+0000","last_became_peered":"2024-04-26T20:02:01.140411+0000","last_unstale":"2024-04-26T20:02:03.239067+0000","last_undegraded":"2024-04-26T20:02:03.239067+0000","last_fullsized":"2024-04-26T20:02:03.239067+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:15.286009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.158278+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.157881+0000","last_became_peered":"2024-04-26T20:02:02.157881+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:08.790750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237528+0000","last_change":"2024-04-26T20:02:01.148686+0000","last_active":"2024-04-26T20:02:03.237528+0000","last_peered":"2024-04-26T20:02:03.237528+0000","last_clean":"2024-04-26T20:02:03.237528+0000","last_became_active":"2024-04-26T20:02:01.148391+0000","last_became_peered":"2024-04-26T20:02:01.148391+0000","last_unstale":"2024-04-26T20:02:03.237528+0000","last_undegraded":"2024-04-26T20:02:03.237528+0000","last_fullsized":"2024-04-26T20:02:03.237528+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.563446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239820+0000","last_change":"2024-04-26T20:02:02.161096+0000","last_active":"2024-04-26T20:02:03.239820+0000","last_peered":"2024-04-26T20:02:03.239820+0000","last_clean":"2024-04-26T20:02:03.239820+0000","last_became_active":"2024-04-26T20:02:02.159993+0000","last_became_peered":"2024-04-26T20:02:02.159993+0000","last_unstale":"2024-04-26T20:02:03.239820+0000","last_undegraded":"2024-04-26T20:02:03.239820+0000","last_fullsized":"2024-04-26T20:02:03.239820+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:40.171693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239709+0000","last_change":"2024-04-26T20:02:01.145418+0000","last_active":"2024-04-26T20:02:03.239709+0000","last_peered":"2024-04-26T20:02:03.239709+0000","last_clean":"2024-04-26T20:02:03.239709+0000","last_became_active":"2024-04-26T20:02:01.145087+0000","last_became_peered":"2024-04-26T20:02:01.145087+0000","last_unstale":"2024-04-26T20:02:03.239709+0000","last_undegraded":"2024-04-26T20:02:03.239709+0000","last_fullsized":"2024-04-26T20:02:03.239709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:52.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236593+0000","last_change":"2024-04-26T20:02:02.154537+0000","last_active":"2024-04-26T20:02:03.236593+0000","last_peered":"2024-04-26T20:02:03.236593+0000","last_clean":"2024-04-26T20:02:03.236593+0000","last_became_active":"2024-04-26T20:02:02.154074+0000","last_became_peered":"2024-04-26T20:02:02.154074+0000","last_unstale":"2024-04-26T20:02:03.236593+0000","last_undegraded":"2024-04-26T20:02:03.236593+0000","last_fullsized":"2024-04-26T20:02:03.236593+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:59.493711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238043+0000","last_change":"2024-04-26T20:02:01.138552+0000","last_active":"2024-04-26T20:02:03.238043+0000","last_peered":"2024-04-26T20:02:03.238043+0000","last_clean":"2024-04-26T20:02:03.238043+0000","last_became_active":"2024-04-26T20:02:01.137755+0000","last_became_peered":"2024-04-26T20:02:01.137755+0000","last_unstale":"2024-04-26T20:02:03.238043+0000","last_undegraded":"2024-04-26T20:02:03.238043+0000","last_fullsized":"2024-04-26T20:02:03.238043+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:29.366170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238605+0000","last_change":"2024-04-26T20:02:02.158357+0000","last_active":"2024-04-26T20:02:03.238605+0000","last_peered":"2024-04-26T20:02:03.238605+0000","last_clean":"2024-04-26T20:02:03.238605+0000","last_became_active":"2024-04-26T20:02:02.158099+0000","last_became_peered":"2024-04-26T20:02:02.158099+0000","last_unstale":"2024-04-26T20:02:03.238605+0000","last_undegraded":"2024-04-26T20:02:03.238605+0000","last_fullsized":"2024-04-26T20:02:03.238605+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:24.333054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146799+0000","last_change":"2024-04-26T20:02:01.141454+0000","last_active":"2024-04-26T20:02:03.146799+0000","last_peered":"2024-04-26T20:02:03.146799+0000","last_clean":"2024-04-26T20:02:03.146799+0000","last_became_active":"2024-04-26T20:02:01.141165+0000","last_became_peered":"2024-04-26T20:02:01.141165+0000","last_unstale":"2024-04-26T20:02:03.146799+0000","last_undegraded":"2024-04-26T20:02:03.146799+0000","last_fullsized":"2024-04-26T20:02:03.146799+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:54.136944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237924+0000","last_change":"2024-04-26T20:02:02.164395+0000","last_active":"2024-04-26T20:02:03.237924+0000","last_peered":"2024-04-26T20:02:03.237924+0000","last_clean":"2024-04-26T20:02:03.237924+0000","last_became_active":"2024-04-26T20:02:02.163976+0000","last_became_peered":"2024-04-26T20:02:02.163976+0000","last_unstale":"2024-04-26T20:02:03.237924+0000","last_undegraded":"2024-04-26T20:02:03.237924+0000","last_fullsized":"2024-04-26T20:02:03.237924+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236377+0000","last_change":"2024-04-26T20:02:01.149337+0000","last_active":"2024-04-26T20:02:03.236377+0000","last_peered":"2024-04-26T20:02:03.236377+0000","last_clean":"2024-04-26T20:02:03.236377+0000","last_became_active":"2024-04-26T20:02:01.147128+0000","last_became_peered":"2024-04-26T20:02:01.147128+0000","last_unstale":"2024-04-26T20:02:03.236377+0000","last_undegraded":"2024-04-26T20:02:03.236377+0000","last_fullsized":"2024-04-26T20:02:03.236377+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:56:50.270032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238079+0000","last_change":"2024-04-26T20:02:02.156728+0000","last_active":"2024-04-26T20:02:03.238079+0000","last_peered":"2024-04-26T20:02:03.238079+0000","last_clean":"2024-04-26T20:02:03.238079+0000","last_became_active":"2024-04-26T20:02:02.156400+0000","last_became_peered":"2024-04-26T20:02:02.156400+0000","last_unstale":"2024-04-26T20:02:03.238079+0000","last_undegraded":"2024-04-26T20:02:03.238079+0000","last_fullsized":"2024-04-26T20:02:03.238079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:29.603442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.147793+0000","last_change":"2024-04-26T20:02:01.130730+0000","last_active":"2024-04-26T20:02:03.147793+0000","last_peered":"2024-04-26T20:02:03.147793+0000","last_clean":"2024-04-26T20:02:03.147793+0000","last_became_active":"2024-04-26T20:02:01.130411+0000","last_became_peered":"2024-04-26T20:02:01.130411+0000","last_unstale":"2024-04-26T20:02:03.147793+0000","last_undegraded":"2024-04-26T20:02:03.147793+0000","last_fullsized":"2024-04-26T20:02:03.147793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:15.214957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238085+0000","last_change":"2024-04-26T20:02:02.162955+0000","last_active":"2024-04-26T20:02:03.238085+0000","last_peered":"2024-04-26T20:02:03.238085+0000","last_clean":"2024-04-26T20:02:03.238085+0000","last_became_active":"2024-04-26T20:02:02.162676+0000","last_became_peered":"2024-04-26T20:02:02.162676+0000","last_unstale":"2024-04-26T20:02:03.238085+0000","last_undegraded":"2024-04-26T20:02:03.238085+0000","last_fullsized":"2024-04-26T20:02:03.238085+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:32.891878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:04.136652+0000","last_change":"2024-04-26T20:02:01.145916+0000","last_active":"2024-04-26T20:02:04.136652+0000","last_peered":"2024-04-26T20:02:04.136652+0000","last_clean":"2024-04-26T20:02:04.136652+0000","last_became_active":"2024-04-26T20:02:01.145659+0000","last_became_peered":"2024-04-26T20:02:01.145659+0000","last_unstale":"2024-04-26T20:02:04.136652+0000","last_undegraded":"2024-04-26T20:02:04.136652+0000","last_fullsized":"2024-04-26T20:02:04.136652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:55:32.895072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146590+0000","last_change":"2024-04-26T20:02:02.148305+0000","last_active":"2024-04-26T20:02:03.146590+0000","last_peered":"2024-04-26T20:02:03.146590+0000","last_clean":"2024-04-26T20:02:03.146590+0000","last_became_active":"2024-04-26T20:02:02.148103+0000","last_became_peered":"2024-04-26T20:02:02.148103+0000","last_unstale":"2024-04-26T20:02:03.146590+0000","last_undegraded":"2024-04-26T20:02:03.146590+0000","last_fullsized":"2024-04-26T20:02:03.146590+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:00.173536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.151161+0000","last_change":"2024-04-26T20:02:01.143779+0000","last_active":"2024-04-26T20:02:03.151161+0000","last_peered":"2024-04-26T20:02:03.151161+0000","last_clean":"2024-04-26T20:02:03.151161+0000","last_became_active":"2024-04-26T20:02:01.143448+0000","last_became_peered":"2024-04-26T20:02:01.143448+0000","last_unstale":"2024-04-26T20:02:03.151161+0000","last_undegraded":"2024-04-26T20:02:03.151161+0000","last_fullsized":"2024-04-26T20:02:03.151161+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:07:53.783522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238528+0000","last_change":"2024-04-26T20:02:02.161591+0000","last_active":"2024-04-26T20:02:03.238528+0000","last_peered":"2024-04-26T20:02:03.238528+0000","last_clean":"2024-04-26T20:02:03.238528+0000","last_became_active":"2024-04-26T20:02:02.160934+0000","last_became_peered":"2024-04-26T20:02:02.160934+0000","last_unstale":"2024-04-26T20:02:03.238528+0000","last_undegraded":"2024-04-26T20:02:03.238528+0000","last_fullsized":"2024-04-26T20:02:03.238528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:05.652569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237874+0000","last_change":"2024-04-26T20:02:01.145371+0000","last_active":"2024-04-26T20:02:03.237874+0000","last_peered":"2024-04-26T20:02:03.237874+0000","last_clean":"2024-04-26T20:02:03.237874+0000","last_became_active":"2024-04-26T20:02:01.145075+0000","last_became_peered":"2024-04-26T20:02:01.145075+0000","last_unstale":"2024-04-26T20:02:03.237874+0000","last_undegraded":"2024-04-26T20:02:03.237874+0000","last_fullsized":"2024-04-26T20:02:03.237874+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:26.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.237103+0000","last_change":"2024-04-26T20:02:02.135333+0000","last_active":"2024-04-26T20:02:03.237103+0000","last_peered":"2024-04-26T20:02:03.237103+0000","last_clean":"2024-04-26T20:02:03.237103+0000","last_became_active":"2024-04-26T20:02:02.135022+0000","last_became_peered":"2024-04-26T20:02:02.135022+0000","last_unstale":"2024-04-26T20:02:03.237103+0000","last_undegraded":"2024-04-26T20:02:03.237103+0000","last_fullsized":"2024-04-26T20:02:03.237103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:17.197831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146631+0000","last_change":"2024-04-26T20:02:01.137606+0000","last_active":"2024-04-26T20:02:03.146631+0000","last_peered":"2024-04-26T20:02:03.146631+0000","last_clean":"2024-04-26T20:02:03.146631+0000","last_became_active":"2024-04-26T20:02:01.137263+0000","last_became_peered":"2024-04-26T20:02:01.137263+0000","last_unstale":"2024-04-26T20:02:03.146631+0000","last_undegraded":"2024-04-26T20:02:03.146631+0000","last_fullsized":"2024-04-26T20:02:03.146631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:43.037846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.239338+0000","last_change":"2024-04-26T20:02:02.166230+0000","last_active":"2024-04-26T20:02:03.239338+0000","last_peered":"2024-04-26T20:02:03.239338+0000","last_clean":"2024-04-26T20:02:03.239338+0000","last_became_active":"2024-04-26T20:02:02.160400+0000","last_became_peered":"2024-04-26T20:02:02.160400+0000","last_unstale":"2024-04-26T20:02:03.239338+0000","last_undegraded":"2024-04-26T20:02:03.239338+0000","last_fullsized":"2024-04-26T20:02:03.239338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:10:54.321252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149809+0000","last_change":"2024-04-26T20:02:01.146319+0000","last_active":"2024-04-26T20:02:03.149809+0000","last_peered":"2024-04-26T20:02:03.149809+0000","last_clean":"2024-04-26T20:02:03.149809+0000","last_became_active":"2024-04-26T20:02:01.145982+0000","last_became_peered":"2024-04-26T20:02:01.145982+0000","last_unstale":"2024-04-26T20:02:03.149809+0000","last_undegraded":"2024-04-26T20:02:03.149809+0000","last_fullsized":"2024-04-26T20:02:03.149809+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:59.976511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146311+0000","last_change":"2024-04-26T20:02:02.161688+0000","last_active":"2024-04-26T20:02:03.146311+0000","last_peered":"2024-04-26T20:02:03.146311+0000","last_clean":"2024-04-26T20:02:03.146311+0000","last_became_active":"2024-04-26T20:02:02.159715+0000","last_became_peered":"2024-04-26T20:02:02.159715+0000","last_unstale":"2024-04-26T20:02:03.146311+0000","last_undegraded":"2024-04-26T20:02:03.146311+0000","last_fullsized":"2024-04-26T20:02:03.146311+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:16.301850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.146475+0000","last_change":"2024-04-26T20:02:01.140827+0000","last_active":"2024-04-26T20:02:03.146475+0000","last_peered":"2024-04-26T20:02:03.146475+0000","last_clean":"2024-04-26T20:02:03.146475+0000","last_became_active":"2024-04-26T20:02:01.140496+0000","last_became_peered":"2024-04-26T20:02:01.140496+0000","last_unstale":"2024-04-26T20:02:03.146475+0000","last_undegraded":"2024-04-26T20:02:03.146475+0000","last_fullsized":"2024-04-26T20:02:03.146475+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.412533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.236398+0000","last_change":"2024-04-26T20:02:02.163701+0000","last_active":"2024-04-26T20:02:03.236398+0000","last_peered":"2024-04-26T20:02:03.236398+0000","last_clean":"2024-04-26T20:02:03.236398+0000","last_became_active":"2024-04-26T20:02:02.162010+0000","last_became_peered":"2024-04-26T20:02:02.162010+0000","last_unstale":"2024-04-26T20:02:03.236398+0000","last_undegraded":"2024-04-26T20:02:03.236398+0000","last_fullsized":"2024-04-26T20:02:03.236398+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:36.989382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235881+0000","last_change":"2024-04-26T20:02:01.142535+0000","last_active":"2024-04-26T20:02:03.235881+0000","last_peered":"2024-04-26T20:02:03.235881+0000","last_clean":"2024-04-26T20:02:03.235881+0000","last_became_active":"2024-04-26T20:02:01.142254+0000","last_became_peered":"2024-04-26T20:02:01.142254+0000","last_unstale":"2024-04-26T20:02:03.235881+0000","last_undegraded":"2024-04-26T20:02:03.235881+0000","last_fullsized":"2024-04-26T20:02:03.235881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:22:17.922754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.235721+0000","last_change":"2024-04-26T20:02:02.142754+0000","last_active":"2024-04-26T20:02:03.235721+0000","last_peered":"2024-04-26T20:02:03.235721+0000","last_clean":"2024-04-26T20:02:03.235721+0000","last_became_active":"2024-04-26T20:02:02.142456+0000","last_became_peered":"2024-04-26T20:02:02.142456+0000","last_unstale":"2024-04-26T20:02:03.235721+0000","last_undegraded":"2024-04-26T20:02:03.235721+0000","last_fullsized":"2024-04-26T20:02:03.235721+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:01.097037+0000","last_clean_scrub_stamp":"2024-04-26T20:02:01.097037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:39.370350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.149884+0000","last_change":"2024-04-26T20:02:01.142278+0000","last_active":"2024-04-26T20:02:03.149884+0000","last_peered":"2024-04-26T20:02:03.149884+0000","last_clean":"2024-04-26T20:02:03.149884+0000","last_became_active":"2024-04-26T20:02:01.141891+0000","last_became_peered":"2024-04-26T20:02:01.141891+0000","last_unstale":"2024-04-26T20:02:03.149884+0000","last_undegraded":"2024-04-26T20:02:03.149884+0000","last_fullsized":"2024-04-26T20:02:03.149884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:00.081196+0000","last_clean_scrub_stamp":"2024-04-26T20:02:00.081196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:27.768532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:02:03.238359+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:02:03.238359+0000","last_peered":"2024-04-26T20:02:03.238359+0000","last_clean":"2024-04-26T20:02:03.238359+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:02:03.238359+0000","last_undegraded":"2024-04-26T20:02:03.238359+0000","last_fullsized":"2024-04-26T20:02:03.238359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1224704,"data_stored":855448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":264920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":265572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":266532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":264976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":626688,"data_stored":265010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:11.977 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:02:11.977 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:02:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.221+0000 7fb5f3200640 1 Processor -- start 2024-04-26T20:02:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.222+0000 7fb5f3200640 1 -- start start 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.223+0000 7fb5f3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 0x7fb5f4073da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.223+0000 7fb5f3200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5f40cd4a0 0x7fb5f415d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.223+0000 7fb5f3200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f415de30 con 0x7fb5f415d9c0 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.223+0000 7fb5f3200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f40d0430 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.223+0000 7fb5f3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f40cd9f0 con 0x7fb5f41466f0 2024-04-26T20:02:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5f40cd4a0 0x7fb5f415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb5f40cd4a0 0x7fb5f415d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42934/0 (socket says 172.21.15.88:42934) 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 -- 172.21.15.88:0/3635086555 learned_addr learned my addr 172.21.15.88:0/3635086555 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1e00640 1 --2- 172.21.15.88:0/3635086555 >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 0x7fb5f4073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 -- 172.21.15.88:0/3635086555 >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 msgr2=0x7fb5f4073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 --2- 172.21.15.88:0/3635086555 >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 0x7fb5f4073da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 -- 172.21.15.88:0/3635086555 >> v1:172.21.15.88:6789/0 conn(0x7fb5f415d9c0 legacy=0x7fb5f4160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1400640 1 -- 172.21.15.88:0/3635086555 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f40cfde0 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f0a00640 1 -- 172.21.15.88:0/3635086555 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010844751 0 0) 0x7fb5f415de30 con 0x7fb5f415d9c0 2024-04-26T20:02:12.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.224+0000 7fb5f1e00640 1 --2- 172.21.15.88:0/3635086555 >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 0x7fb5f4073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:12.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.225+0000 7fb5f1400640 1 --2- 172.21.15.88:0/3635086555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f40cd4a0 0x7fb5f415d480 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40743e0 tx=0x7fb5e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a279ec065b07ff95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f0a00640 1 -- 172.21.15.88:0/3635086555 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4037070 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f0a00640 1 -- 172.21.15.88:0/3635086555 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e4032440 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f0a00640 1 -- 172.21.15.88:0/3635086555 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4032810 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 -- 172.21.15.88:0/3635086555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f40cd4a0 msgr2=0x7fb5f415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3635086555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f40cd4a0 0x7fb5f415d480 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40743e0 tx=0x7fb5e402a270 comp rx=0 tx=0).stop 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 -- 172.21.15.88:0/3635086555 shutdown_connections 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3635086555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f40cd4a0 0x7fb5f415d480 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3635086555 >> v2:172.21.15.132:3300/0 conn(0x7fb5f41466f0 0x7fb5f4073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 -- 172.21.15.88:0/3635086555 >> 172.21.15.88:0/3635086555 conn(0x7fb5f40b2790 msgr2=0x7fb5f40b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 -- 172.21.15.88:0/3635086555 shutdown_connections 2024-04-26T20:02:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.226+0000 7fb5f3200640 1 -- 172.21.15.88:0/3635086555 wait complete. 2024-04-26T20:02:12.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.228+0000 7fb5f3200640 1 Processor -- start 2024-04-26T20:02:12.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.228+0000 7fb5f3200640 1 -- start start 2024-04-26T20:02:12.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5f40cd4a0 0x7fb5f40b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f41466f0 0x7fb5f415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4146ad0 con 0x7fb5f415d9c0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f415e320 con 0x7fb5f41466f0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.229+0000 7fb5f3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f4146c50 con 0x7fb5f40cd4a0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42950/0 (socket says 172.21.15.88:42950) 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 -- 172.21.15.88:0/3586213443 learned_addr learned my addr 172.21.15.88:0/3586213443 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f1400640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f41466f0 0x7fb5f415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f1e00640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5f40cd4a0 0x7fb5f40b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 -- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5f40cd4a0 msgr2=0x7fb5f40b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5f40cd4a0 0x7fb5f40b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 -- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f41466f0 msgr2=0x7fb5f415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f41466f0 0x7fb5f415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 -- 172.21.15.88:0/3586213443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f40b5bd0 con 0x7fb5f415d9c0 2024-04-26T20:02:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.230+0000 7fb5f2800640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c780 tx=0x7fb5ec00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.231+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003b70 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.231+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ec003d10 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.231+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec005270 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.231+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f40740e0 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.231+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f40b0140 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.232+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f4050760 con 0x7fb5f415d9c0 2024-04-26T20:02:12.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.233+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5ec005410 con 0x7fb5f415d9c0 2024-04-26T20:02:12.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.234+0000 7fb5ea000640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.234+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ec08ba10 con 0x7fb5f415d9c0 2024-04-26T20:02:12.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.234+0000 7fb5f1e00640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.235+0000 7fb5f1e00640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 0x7fb5d006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a990 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.238+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5ec059e30 con 0x7fb5f415d9c0 2024-04-26T20:02:12.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.513+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb5f4161980 con 0x7fb5f415d9c0 2024-04-26T20:02:12.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.517+0000 7fb5ea000640 1 -- 172.21.15.88:0/3586213443 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fb5ec059730 con 0x7fb5f415d9c0 2024-04-26T20:02:12.518 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-04-26T20:02:12.519 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw== 2024-04-26T20:02:12.519 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-04-26T20:02:12.519 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-04-26T20:02:12.519 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:02:12.519 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-04-26T20:02:12.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 msgr2=0x7fb5d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 0x7fb5d006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a990 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2024-04-26T20:02:12.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 msgr2=0x7fb5f4073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00c780 tx=0x7fb5ec00cb80 comp rx=0 tx=0).stop 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 shutdown_connections 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5f415d9c0 0x7fb5f4073ba0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5f41466f0 0x7fb5f415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb5d006c920 0x7fb5d006ede0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 --2- 172.21.15.88:0/3586213443 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb5f40cd4a0 0x7fb5f40b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 >> 172.21.15.88:0/3586213443 conn(0x7fb5f40b2790 msgr2=0x7fb5f41421f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 shutdown_connections 2024-04-26T20:02:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.520+0000 7fb5f3200640 1 -- 172.21.15.88:0/3586213443 wait complete. 2024-04-26T20:02:12.539 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:12.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.732+0000 7f1a49800640 1 Processor -- start 2024-04-26T20:02:12.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.732+0000 7f1a49800640 1 -- start start 2024-04-26T20:02:12.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.733+0000 7f1a49800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1a4406ce80 0x7f1a440add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.733+0000 7f1a49800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a440cd650 0x7f1a440b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.733+0000 7f1a49800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a440af9a0 con 0x7f1a4415cdf0 2024-04-26T20:02:12.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a49800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1a440ae2c0 con 0x7f1a4406ce80 2024-04-26T20:02:12.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a49800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a440b05b0 con 0x7f1a440cd650 2024-04-26T20:02:12.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1a4406ce80 0x7f1a440add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1a4406ce80 0x7f1a440add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42964/0 (socket says 172.21.15.88:42964) 2024-04-26T20:02:12.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 -- 172.21.15.88:0/2867815631 learned_addr learned my addr 172.21.15.88:0/2867815631 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a43400640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a440cd650 0x7f1a440b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 -- 172.21.15.88:0/2867815631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a440cd650 msgr2=0x7f1a440b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a440cd650 0x7f1a440b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 -- 172.21.15.88:0/2867815631 >> v1:172.21.15.132:6789/0 conn(0x7f1a4415cdf0 legacy=0x7f1a4406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.734+0000 7f1a42000640 1 -- 172.21.15.88:0/2867815631 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a440afc80 con 0x7f1a4406ce80 2024-04-26T20:02:12.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.735+0000 7f1a42000640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a440add80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009870 tx=0x7f1a3002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed0787255e83984 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a41600640 1 -- 172.21.15.88:0/2867815631 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30037070 con 0x7f1a4406ce80 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a41600640 1 -- 172.21.15.88:0/2867815631 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30032440 con 0x7f1a4406ce80 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a41600640 1 -- 172.21.15.88:0/2867815631 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30032810 con 0x7f1a4406ce80 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 -- 172.21.15.88:0/2867815631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 msgr2=0x7f1a440add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a440add80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1a30009870 tx=0x7f1a3002a210 comp rx=0 tx=0).stop 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 -- 172.21.15.88:0/2867815631 shutdown_connections 2024-04-26T20:02:12.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a440cd650 0x7f1a440b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 --2- 172.21.15.88:0/2867815631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a440add80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 -- 172.21.15.88:0/2867815631 >> 172.21.15.88:0/2867815631 conn(0x7f1a440b2d50 msgr2=0x7f1a440b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:12.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 -- 172.21.15.88:0/2867815631 shutdown_connections 2024-04-26T20:02:12.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.736+0000 7f1a49800640 1 -- 172.21.15.88:0/2867815631 wait complete. 2024-04-26T20:02:12.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.738+0000 7f1a49800640 1 Processor -- start 2024-04-26T20:02:12.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.738+0000 7f1a49800640 1 -- start start 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 0x7f1a440b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a4415cdf0 0x7f1a4414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4415d1d0 con 0x7f1a4406ce80 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440b0240 con 0x7f1a440cd650 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a49800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4415d350 con 0x7f1a4415cdf0 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a42a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a42a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42976/0 (socket says 172.21.15.88:42976) 2024-04-26T20:02:12.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.739+0000 7f1a42a00640 1 -- 172.21.15.88:0/1124481905 learned_addr learned my addr 172.21.15.88:0/1124481905 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42000640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 0x7f1a440b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 -- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a4415cdf0 msgr2=0x7f1a4414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a43400640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a4415cdf0 0x7f1a4414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a4415cdf0 0x7f1a4414d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 -- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 msgr2=0x7f1a440b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 0x7f1a440b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 -- 172.21.15.88:0/1124481905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a440b24c0 con 0x7f1a4406ce80 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42000640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 0x7f1a440b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a42a00640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c460 tx=0x7f1a3400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34016020 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.740+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a34012870 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.741+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34012c60 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.741+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a44153550 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.741+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4414f4c0 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.742+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44069540 con 0x7f1a4406ce80 2024-04-26T20:02:12.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.743+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a34019020 con 0x7f1a4406ce80 2024-04-26T20:02:12.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.744+0000 7f1a3aa00640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:12.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.744+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f1a3408b920 con 0x7f1a4406ce80 2024-04-26T20:02:12.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.744+0000 7f1a42000640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:12.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.745+0000 7f1a42000640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 0x7f1a2406edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1a30034000 tx=0x7f1a3002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:12.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.749+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a34020030 con 0x7f1a4406ce80 2024-04-26T20:02:12.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.989+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a440d1930 con 0x7f1a4406ce80 2024-04-26T20:02:12.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.991+0000 7f1a3aa00640 1 -- 172.21.15.88:0/1124481905 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5181 (secure 0 0 0) 0x7f1a34059cc0 con 0x7f1a4406ce80 2024-04-26T20:02:12.993 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:12.993 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:12.994 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 29 2024-04-26T20:02:12.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 msgr2=0x7f1a2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 0x7f1a2406edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1a30034000 tx=0x7f1a3002e000 comp rx=0 tx=0).stop 2024-04-26T20:02:12.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 msgr2=0x7f1a44152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:12.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c460 tx=0x7f1a3400c920 comp rx=0 tx=0).stop 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 shutdown_connections 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1a2406c8f0 0x7f1a2406edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1a4415cdf0 0x7f1a4414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440cd650 0x7f1a440b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 --2- 172.21.15.88:0/1124481905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a4406ce80 0x7f1a44152f70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.994+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 >> 172.21.15.88:0/1124481905 conn(0x7f1a440b2d50 msgr2=0x7f1a440727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:12.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.995+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 shutdown_connections 2024-04-26T20:02:12.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:12.995+0000 7f1a49800640 1 -- 172.21.15.88:0/1124481905 wait complete. 2024-04-26T20:02:13.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:02:01.583597+0000', 'modified': '2024-04-26T20:02:04.646446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5712': {'gid': 5712, 'name': 'a', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6837/649375356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 649375356}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 649375356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5712, 'qdb_cluster': [5712]} 2024-04-26T20:02:13.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:02:13.421 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T20:02:13.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.633+0000 7f6589e00640 1 Processor -- start 2024-04-26T20:02:13.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.634+0000 7f6589e00640 1 -- start start 2024-04-26T20:02:13.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.635+0000 7f6589e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 0x7f658406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.635+0000 7f6589e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658415cfd0 0x7f658406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.635+0000 7f6589e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65840ae850 con 0x7f658406c770 2024-04-26T20:02:13.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.635+0000 7f6589e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f65840af460 con 0x7f658415cfd0 2024-04-26T20:02:13.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.635+0000 7f6589e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f65840aeb30 con 0x7f658415cb70 2024-04-26T20:02:13.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658415cfd0 0x7f658406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f658415cfd0 0x7f658406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42986/0 (socket says 172.21.15.88:42986) 2024-04-26T20:02:13.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 -- 172.21.15.88:0/744522463 learned_addr learned my addr 172.21.15.88:0/744522463 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f657be00640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 0x7f658406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 -- 172.21.15.88:0/744522463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 msgr2=0x7f658406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 0x7f658406ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6582000640 1 -- 172.21.15.88:0/744522463 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567482236 0 0) 0x7f65840ae850 con 0x7f658406c770 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 -- 172.21.15.88:0/744522463 >> v1:172.21.15.88:6789/0 conn(0x7f658406c770 legacy=0x7f658406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f6583400640 1 -- 172.21.15.88:0/744522463 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65840aee10 con 0x7f658415cfd0 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.636+0000 7f657be00640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 0x7f658406ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.637+0000 7f6583400640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cfd0 0x7f658406f3a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d150 tx=0x7f657c02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcd323cb2072580 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6582000640 1 -- 172.21.15.88:0/744522463 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c009600 con 0x7f658415cfd0 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6582000640 1 -- 172.21.15.88:0/744522463 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c037070 con 0x7f658415cfd0 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6582000640 1 -- 172.21.15.88:0/744522463 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0099c0 con 0x7f658415cfd0 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 -- 172.21.15.88:0/744522463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cfd0 msgr2=0x7f658406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cfd0 0x7f658406f3a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d150 tx=0x7f657c02c930 comp rx=0 tx=0).stop 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 -- 172.21.15.88:0/744522463 shutdown_connections 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cfd0 0x7f658406f3a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 --2- 172.21.15.88:0/744522463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cb70 0x7f658406ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 -- 172.21.15.88:0/744522463 >> 172.21.15.88:0/744522463 conn(0x7f65840b2d50 msgr2=0x7f65840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 -- 172.21.15.88:0/744522463 shutdown_connections 2024-04-26T20:02:13.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.638+0000 7f6589e00640 1 -- 172.21.15.88:0/744522463 wait complete. 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.640+0000 7f6589e00640 1 Processor -- start 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.640+0000 7f6589e00640 1 -- start start 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f658406c770 0x7f6584147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cfd0 0x7f65840ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f658415c780 con 0x7f658415cb70 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f658415c600 con 0x7f658406c770 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f6589e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f65840af8f0 con 0x7f658415cfd0 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f657be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f657be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43000/0 (socket says 172.21.15.88:43000) 2024-04-26T20:02:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.641+0000 7f657be00640 1 -- 172.21.15.88:0/6276610 learned_addr learned my addr 172.21.15.88:0/6276610 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 -- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cfd0 msgr2=0x7f65840ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cfd0 0x7f65840ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 -- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f658406c770 msgr2=0x7f6584147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f658406c770 0x7f6584147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 -- 172.21.15.88:0/6276610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584073080 con 0x7f658415cb70 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657be00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c7f0 tx=0x7f656c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657b400640 1 -- 172.21.15.88:0/6276610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c015070 con 0x7f658415cb70 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657b400640 1 -- 172.21.15.88:0/6276610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c007a30 con 0x7f658415cb70 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6584073360 con 0x7f658415cb70 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.642+0000 7f657b400640 1 -- 172.21.15.88:0/6276610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c007e00 con 0x7f658415cb70 2024-04-26T20:02:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.643+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584073920 con 0x7f658415cb70 2024-04-26T20:02:13.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.645+0000 7f657b400640 1 -- 172.21.15.88:0/6276610 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f656c018070 con 0x7f658415cb70 2024-04-26T20:02:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.645+0000 7f657b400640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 0x7f656006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.646+0000 7f657b400640 1 -- 172.21.15.88:0/6276610 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f656c08bbc0 con 0x7f658415cb70 2024-04-26T20:02:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.646+0000 7f6582a00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 0x7f656006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.647+0000 7f6582a00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 0x7f656006edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6574001600 tx=0x7f6574007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.647+0000 7f658cc1d740 1 Processor -- start 2024-04-26T20:02:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.647+0000 7f658cc1d740 1 -- start start 2024-04-26T20:02:13.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b046dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55de7afa6600 con 0x55de7b0469f0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55de7af43100 con 0x55de7b0830c0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55de7afa25c0 con 0x55de7b0714b0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43012/0 (socket says 172.21.15.88:43012) 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 -- 172.21.15.88:0/4276956931 learned_addr learned my addr 172.21.15.88:0/4276956931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 -- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 msgr2=0x55de7b071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 -- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 msgr2=0x55de7b046dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f657be00640 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b046dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b046dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.648+0000 7f6582a00640 1 -- 172.21.15.88:0/4276956931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7acb82d0 con 0x55de7b0469f0 2024-04-26T20:02:13.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.649+0000 7f6582a00640 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6574005550 tx=0x7f657400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924dc920186c33c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.649+0000 7f6573400640 1 -- 172.21.15.88:0/4276956931 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574016510 con 0x55de7b0469f0 2024-04-26T20:02:13.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.649+0000 7f6573400640 1 -- 172.21.15.88:0/4276956931 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65740166d0 con 0x55de7b0469f0 2024-04-26T20:02:13.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.649+0000 7f6573400640 1 -- 172.21.15.88:0/4276956931 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574016ac0 con 0x55de7b0469f0 2024-04-26T20:02:13.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- 172.21.15.88:0/4276956931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 msgr2=0x55de7b048b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6574005550 tx=0x7f657400f000 comp rx=0 tx=0).stop 2024-04-26T20:02:13.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- 172.21.15.88:0/4276956931 shutdown_connections 2024-04-26T20:02:13.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b046dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4276956931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b048b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- 172.21.15.88:0/4276956931 >> 172.21.15.88:0/4276956931 conn(0x55de7afd6d10 msgr2=0x55de7b08c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- 172.21.15.88:0/4276956931 shutdown_connections 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- 172.21.15.88:0/4276956931 wait complete. 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.650+0000 7f658cc1d740 1 -- start start 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.651+0000 7f658cc1d740 1 Processor -- start 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0469f0 0x55de7b0851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b072df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55de7af2df70 con 0x55de7b0830c0 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55de7b06f000 con 0x55de7b0469f0 2024-04-26T20:02:13.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55de7b09d600 con 0x55de7b0714b0 2024-04-26T20:02:13.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f658cc1d740 10 client.0 _ll_register_callbacks cb 0x55de7af9c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:02:13.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43016/0 (socket says 172.21.15.88:43016) 2024-04-26T20:02:13.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 -- 172.21.15.88:0/3426781600 learned_addr learned my addr 172.21.15.88:0/3426781600 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6582a00640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0469f0 0x55de7b0851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f657be00640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 -- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 msgr2=0x55de7b072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 -- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0469f0 msgr2=0x55de7b0851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0469f0 0x55de7b0851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.652+0000 7f6583400640 1 -- 172.21.15.88:0/3426781600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7ad826b0 con 0x55de7b0830c0 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f6583400640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d150 tx=0x7f657c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8ceb8dd3488e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f6567e00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:02:13.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c032830 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f657c0329d0 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65740166d0 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f657c04d050 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f658cc1d740 10 client.5871 fetch_fsmap learned FSMap version 29 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.653+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55de7b09d340 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65740166d0 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f657c050020 con 0x55de7b0830c0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 10 client.5871 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 10 client.5871 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 20 client.5871 populate_metadata read hostname 'smithi088' 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b071930 0x55de7b09c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 4 client.5871 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 --> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de7afb4210 con 0x55de7b071930 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.654+0000 7f6582a00640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b071930 0x55de7b09c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.655+0000 7f6582a00640 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b071930 0x55de7b09c3b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2813aaa7a603b753 server_cookie=b2fe5d5c67501708 in_seq=0 out_seq=0 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.655+0000 7f6567e00640 10 client.5871 ms_handle_connect on v2:172.21.15.88:6836/649375356 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.655+0000 7f6567e00640 1 -- 172.21.15.88:0/3426781600 <== mds.0 v2:172.21.15.88:6836/649375356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de7afb4210 con 0x55de7b071930 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.655+0000 7f6567e00640 10 client.5871 handle_command_reply: tid=0 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 client.5871 shutdown 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 msgr2=0x55de7b06ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f657c00d150 tx=0x7f657c00b510 comp rx=0 tx=0).stop 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 shutdown_connections 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b071930 0x55de7b09c3b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0830c0 0x55de7b06ca00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 --2- 172.21.15.88:0/3426781600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0469f0 0x55de7b0851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 >> 172.21.15.88:0/3426781600 conn(0x55de7afd6d10 msgr2=0x55de7b08fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f6567e00640 0 client.5871 ms_handle_reset on v2:172.21.15.88:6836/649375356 2024-04-26T20:02:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 shutdown_connections 2024-04-26T20:02:13.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 1 -- 172.21.15.88:0/3426781600 wait complete. 2024-04-26T20:02:13.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.656+0000 7f658cc1d740 20 client.5871 trim_cache size 0 max 16384 2024-04-26T20:02:13.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.676+0000 7f658cc1d740 1 Processor -- start 2024-04-26T20:02:13.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.676+0000 7f658cc1d740 1 -- start start 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b0727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f658409cba0 con 0x55de7b0469f0 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f658415a730 con 0x55de7b0830c0 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f658409c550 con 0x55de7b0714b0 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43026/0 (socket says 172.21.15.88:43026) 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 -- 172.21.15.88:0/4112793890 learned_addr learned my addr 172.21.15.88:0/4112793890 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 -- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 msgr2=0x55de7b0727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b0727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 -- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 msgr2=0x55de7b072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.677+0000 7f6582a00640 1 -- 172.21.15.88:0/4112793890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7b085530 con 0x55de7b0469f0 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.678+0000 7f6582a00640 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6574015000 tx=0x7f6574005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56a435e71342fce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.678+0000 7f6572a00640 1 -- 172.21.15.88:0/4112793890 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657402b3c0 con 0x55de7b0469f0 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.678+0000 7f6572a00640 1 -- 172.21.15.88:0/4112793890 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657402b560 con 0x55de7b0469f0 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.678+0000 7f6572a00640 1 -- 172.21.15.88:0/4112793890 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657402b910 con 0x55de7b0469f0 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- 172.21.15.88:0/4112793890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 msgr2=0x55de7b075000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6574015000 tx=0x7f6574005000 comp rx=0 tx=0).stop 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- 172.21.15.88:0/4112793890 shutdown_connections 2024-04-26T20:02:13.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0830c0 0x55de7b072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0714b0 0x55de7b0727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 --2- 172.21.15.88:0/4112793890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b075000 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- 172.21.15.88:0/4112793890 >> 172.21.15.88:0/4112793890 conn(0x55de7afd6d10 msgr2=0x55de7b038d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- 172.21.15.88:0/4112793890 shutdown_connections 2024-04-26T20:02:13.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- 172.21.15.88:0/4112793890 wait complete. 2024-04-26T20:02:13.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.679+0000 7f658cc1d740 1 -- start start 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.680+0000 7f658cc1d740 1 Processor -- start 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 0x55de7b0844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 0x55de7b0457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55de7b06d3f0 con 0x55de7b0469f0 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55de7b06cf40 con 0x55de7b0714b0 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55de7b071db0 con 0x55de7b0830c0 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f658cc1d740 10 client.0 _ll_register_callbacks cb 0x7f657c00d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6582a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43032/0 (socket says 172.21.15.88:43032) 2024-04-26T20:02:13.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6582a00640 1 -- 172.21.15.88:0/232783114 learned_addr learned my addr 172.21.15.88:0/232783114 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f657be00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 0x55de7b0844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6582a00640 1 -- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 msgr2=0x55de7b0457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6583400640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 0x55de7b0457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.681+0000 7f6582a00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 0x55de7b0457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6582a00640 1 -- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 msgr2=0x55de7b0844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6582a00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 0x55de7b0844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6582a00640 1 -- 172.21.15.88:0/232783114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7b08f360 con 0x55de7b0469f0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f657be00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 0x55de7b0844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6583400640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 0x55de7b0457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6582a00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6574023040 tx=0x7f65740010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d380ffd429b6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6567e00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657402b920 con 0x55de7b0469f0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f6574007050 con 0x55de7b0469f0 2024-04-26T20:02:13.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657402b560 con 0x55de7b0469f0 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f657403b050 con 0x55de7b0469f0 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f658cc1d740 10 client.5877 fetch_fsmap learned FSMap version 29 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.682+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55de7b086890 con 0x55de7b0469f0 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657402b560 con 0x55de7b0469f0 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f657403e020 con 0x55de7b0469f0 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 10 client.5877 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 10 client.5877 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 20 client.5877 populate_metadata read hostname 'smithi088' 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b105ff0 0x55de7b081e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 4 client.5877 mds_command: new command op to 5712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:02:13.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 --> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] -- command(tid 0: {"prefix": "status"}) -- 0x55de7afae080 con 0x55de7b105ff0 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.683+0000 7f657be00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b105ff0 0x55de7b081e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f657be00640 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b105ff0 0x55de7b081e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819160dc5898e40a server_cookie=57951c321e891ce3 in_seq=0 out_seq=0 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f6567e00640 10 client.5877 ms_handle_connect on v2:172.21.15.88:6836/649375356 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f6567e00640 1 -- 172.21.15.88:0/232783114 <== mds.0 v2:172.21.15.88:6836/649375356 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55de7afae080 con 0x55de7b105ff0 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f6567e00640 10 client.5877 handle_command_reply: tid=0 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 client.5877 shutdown 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 msgr2=0x55de7b083f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6574023040 tx=0x7f65740010f0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 shutdown_connections 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:6836/649375356,v1:172.21.15.88:6837/649375356] conn(0x55de7b105ff0 0x55de7b081e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55de7b0830c0 0x55de7b0457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55de7b0714b0 0x55de7b0844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 --2- 172.21.15.88:0/232783114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de7b0469f0 0x55de7b083f80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f6567e00640 0 client.5877 ms_handle_reset on v2:172.21.15.88:6836/649375356 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 >> 172.21.15.88:0/232783114 conn(0x55de7afd6d10 msgr2=0x55de7b072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.684+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 shutdown_connections 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f658cc1d740 1 -- 172.21.15.88:0/232783114 wait complete. 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f658cc1d740 20 client.5877 trim_cache size 0 max 16384 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "100f2996-174e-4373-bd56-473fe683ba7a", 2024-04-26T20:02:13.692 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5712, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.568202466000001, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 29, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 52, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 52, 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 17.661409665000001 2024-04-26T20:02:13.693 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 msgr2=0x7f656006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 0x7f656006edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6574001600 tx=0x7f6574007910 comp rx=0 tx=0).stop 2024-04-26T20:02:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 msgr2=0x7f65840ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.685+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c7f0 tx=0x7f656c00cbf0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 shutdown_connections 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f656006c8f0 0x7f656006edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f658415cfd0 0x7f65840ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f658415cb70 0x7f65840ce290 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 --2- 172.21.15.88:0/6276610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f658406c770 0x7f6584147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 >> 172.21.15.88:0/6276610 conn(0x7f65840b2d50 msgr2=0x7f658406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 shutdown_connections 2024-04-26T20:02:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.686+0000 7f6589e00640 1 -- 172.21.15.88:0/6276610 wait complete. 2024-04-26T20:02:13.705 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:02:13.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.893+0000 7f341ec00640 1 Processor -- start 2024-04-26T20:02:13.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.893+0000 7f341ec00640 1 -- start start 2024-04-26T20:02:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341ec00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341815f1e0 0x7f34180b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34180d3190 0x7f34180b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341ec00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f341815f5c0 con 0x7f34180d5b60 2024-04-26T20:02:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341ec00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f34180b8670 con 0x7f341815f1e0 2024-04-26T20:02:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f34180b8950 con 0x7f34180d3190 2024-04-26T20:02:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.895+0000 7f341d800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341815f1e0 0x7f34180b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f341815f1e0 0x7f34180b7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:52072/0 (socket says 172.21.15.88:52072) 2024-04-26T20:02:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 -- 172.21.15.88:0/3885996770 learned_addr learned my addr 172.21.15.88:0/3885996770 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341e200640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34180d3190 0x7f34180b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 -- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34180d3190 msgr2=0x7f34180b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34180d3190 0x7f34180b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 -- 172.21.15.88:0/3885996770 >> v1:172.21.15.132:6789/0 conn(0x7f34180d5b60 legacy=0x7f34180d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.896+0000 7f341d800640 1 -- 172.21.15.88:0/3885996770 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180b8c30 con 0x7f341815f1e0 2024-04-26T20:02:13.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.898+0000 7f341d800640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341815f1e0 0x7f34180b7d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f340c009870 tx=0x7f340c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c0cfeb6c21bb183 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.898+0000 7f3417e00640 1 -- 172.21.15.88:0/3885996770 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c037070 con 0x7f341815f1e0 2024-04-26T20:02:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.898+0000 7f3417e00640 1 -- 172.21.15.88:0/3885996770 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c032440 con 0x7f341815f1e0 2024-04-26T20:02:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.898+0000 7f3417e00640 1 -- 172.21.15.88:0/3885996770 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c032810 con 0x7f341815f1e0 2024-04-26T20:02:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 -- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341815f1e0 msgr2=0x7f34180b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341815f1e0 0x7f34180b7d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f340c009870 tx=0x7f340c02a270 comp rx=0 tx=0).stop 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 -- 172.21.15.88:0/3885996770 shutdown_connections 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34180d3190 0x7f34180b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 --2- 172.21.15.88:0/3885996770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f341815f1e0 0x7f34180b7d90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 -- 172.21.15.88:0/3885996770 >> 172.21.15.88:0/3885996770 conn(0x7f341809e2d0 msgr2=0x7f34180d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 -- 172.21.15.88:0/3885996770 shutdown_connections 2024-04-26T20:02:13.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.899+0000 7f341ec00640 1 -- 172.21.15.88:0/3885996770 wait complete. 2024-04-26T20:02:13.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.901+0000 7f341ec00640 1 Processor -- start 2024-04-26T20:02:13.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.901+0000 7f341ec00640 1 -- start start 2024-04-26T20:02:13.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34180d3190 0x7f341814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341815f1e0 0x7f3418149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34180b82d0 con 0x7f34180d5b60 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f34180b91f0 con 0x7f34180d3190 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f34180b8450 con 0x7f341815f1e0 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43040/0 (socket says 172.21.15.88:43040) 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.902+0000 7f341ce00640 1 -- 172.21.15.88:0/810556045 learned_addr learned my addr 172.21.15.88:0/810556045 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341d800640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34180d3190 0x7f341814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341e200640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341815f1e0 0x7f3418149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 -- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341815f1e0 msgr2=0x7f3418149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341815f1e0 0x7f3418149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 -- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34180d3190 msgr2=0x7f341814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34180d3190 0x7f341814c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 -- 172.21.15.88:0/810556045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180c9ab0 con 0x7f34180d5b60 2024-04-26T20:02:13.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ce00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f340400c400 tx=0x7f340400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040052d0 con 0x7f34180d5b60 2024-04-26T20:02:13.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3404005490 con 0x7f34180d5b60 2024-04-26T20:02:13.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404007260 con 0x7f34180d5b60 2024-04-26T20:02:13.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341814cc70 con 0x7f34180d5b60 2024-04-26T20:02:13.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.903+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341814d230 con 0x7f34180d5b60 2024-04-26T20:02:13.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.905+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34180693c0 con 0x7f34180d5b60 2024-04-26T20:02:13.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.905+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3404007400 con 0x7f34180d5b60 2024-04-26T20:02:13.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.906+0000 7f3415600640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 0x7f33f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:13.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.906+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f340408b140 con 0x7f34180d5b60 2024-04-26T20:02:13.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.907+0000 7f341d800640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 0x7f33f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:13.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.907+0000 7f341d800640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 0x7f33f806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f340c034000 tx=0x7f340c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:13.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:13.911+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34040594d0 con 0x7f34180d5b60 2024-04-26T20:02:14.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.150+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f34180b6f40 con 0x7f34180d5b60 2024-04-26T20:02:14.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.151+0000 7f3415600640 1 -- 172.21.15.88:0/810556045 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7f3404058dd0 con 0x7f34180d5b60 2024-04-26T20:02:14.153 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 msgr2=0x7f33f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 0x7f33f806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f340c034000 tx=0x7f340c02e000 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 msgr2=0x7f34181f5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f340400c400 tx=0x7f340400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 shutdown_connections 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f33f806c8f0 0x7f33f806edb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f341815f1e0 0x7f3418149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34180d5b60 0x7f34181f5550 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 --2- 172.21.15.88:0/810556045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34180d3190 0x7f341814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 >> 172.21.15.88:0/810556045 conn(0x7f341809e2d0 msgr2=0x7f34180d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 shutdown_connections 2024-04-26T20:02:14.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14.154+0000 7f341ec00640 1 -- 172.21.15.88:0/810556045 wait complete. 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:02:14.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:02:14.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:02:14.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:14.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.176 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:02:14.225 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-04-26T20:02:14.226 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::9c6e:acff:fec3:49a/64 scope link 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:33:4d:d7:93:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c433:4dff:fed7:93b6/64 scope link 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:14.227 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::2caf:4ff:fe1f:b3fa/64 scope link 2024-04-26T20:02:14.228 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:14.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.228 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:02:14.303 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-04-26T20:02:14.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.307 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-04-26T20:02:14.361 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-04-26T20:02:14.361 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:02:14.361 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:02:14.361 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:02:14.361 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:02:14.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:02:14.362 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:02:14.362 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:02:14.362 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:02:14.362 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:02:14.362 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:02:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:02:14.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:14.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.571 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:02:14.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:02:14.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:02:14.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:14.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-04-26T20:02:14.701 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:02:14.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:14.702 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:14.757 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:14.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:14.757 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:14.808 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:02:14.892 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:14.948 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:02:14.948 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:02:14.948 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:02:14.948 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:02:14.949 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:02:14.950 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:02:14.951 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:02:14.951 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:14.951 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:14.951 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:14.951 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:14.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:14.952 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:02:15.002 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:02:15.002 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:02:15.009 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:02:15.077 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:15.122 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:02:15.122 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:02:15.122 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:02:15.122 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:02:15.123 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:02:15.124 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:02:15.124 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:02:15.124 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:15.125 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:02:15.126 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:02:15.127 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:15.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:15.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:15.128 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:15.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:15.130 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:02:15.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2024-04-26T20:02:15.176+0000 7f7eb951e480 -1 init, newargv = 0x560a62a4cd50 newargc=15 2024-04-26T20:02:15.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50562]: starting ceph client 2024-04-26T20:02:15.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50562]: starting fuse 2024-04-26T20:02:15.211 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:02:15.211 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:02:15.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:02:15.211 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-04-26T20:02:15.212 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2024-04-26T20:02:15.213 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-04-26T20:02:15.396 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.50562.asok 2024-04-26T20:02:15.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:15.413 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50562.asok 2024-04-26T20:02:15.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:15.413 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50562.asok status 2024-04-26T20:02:15.641 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "50562", 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5889, 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-04-26T20:02:15.642 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 5889 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3921998306 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3921998306 2024-04-26T20:02:15.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.5889 192.168.0.1:0/3921998306", 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/3921998306", 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 29, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 52, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-04-26T20:02:15.644 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:15.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:15.656 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-04-26T20:02:15.656 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:15.708 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-04-26T20:02:15.708 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:15.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:15.708 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:15.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:02:15.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.994+0000 7f3f28a00640 1 Processor -- start 2024-04-26T20:02:15.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.994+0000 7f3f28a00640 1 -- start start 2024-04-26T20:02:15.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f28a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3f2415dc80 0x7f3f24167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:15.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f28a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f24164740 con 0x7f3f2406c770 2024-04-26T20:02:15.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f28a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f241671a0 con 0x7f3f24164240 2024-04-26T20:02:15.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f28a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3f240b5ff0 con 0x7f3f2415dc80 2024-04-26T20:02:15.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f21400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3f2406c770 0x7f3f24163b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40476/0 (socket says 172.21.15.88:40476) 2024-04-26T20:02:15.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f21400640 1 -- 172.21.15.88:0/3288392826 learned_addr learned my addr 172.21.15.88:0/3288392826 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f21e00640 1 --2- 172.21.15.88:0/3288392826 >> v2:172.21.15.132:3300/0 conn(0x7f3f2415dc80 0x7f3f24167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420999403 0 0) 0x7f3f24164740 con 0x7f3f2406c770 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.996+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f08003520 con 0x7f3f2406c770 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994585866 0 0) 0x7f3f08003520 con 0x7f3f2406c770 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 >> v2:172.21.15.132:3300/0 conn(0x7f3f2415dc80 msgr2=0x7f3f24167c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 --2- 172.21.15.88:0/3288392826 >> v2:172.21.15.132:3300/0 conn(0x7f3f2415dc80 0x7f3f24167c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 >> v1:172.21.15.132:6789/0 conn(0x7f3f24164240 legacy=0x7f3f240add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f24167480 con 0x7f3f2406c770 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f3f10001690 con 0x7f3f2406c770 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 >> v1:172.21.15.88:6789/0 conn(0x7f3f2406c770 legacy=0x7f3f24163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f08003a00 0x7f3f08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800c110 con 0x7f3f08005210 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800c2c0 con 0x7f3f08004800 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0800c4c0 con 0x7f3f08003a00 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.997+0000 7f3f22800640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f21e00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f08003a00 0x7f3f08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f21400640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f22800640 1 -- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f08003a00 msgr2=0x7f3f08003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f22800640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f08003a00 0x7f3f08003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f22800640 1 -- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f08004800 msgr2=0x7f3f08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f22800640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.998+0000 7f3f22800640 1 -- 172.21.15.88:0/3288392826 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f08003520 con 0x7f3f08005210 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.999+0000 7f3f22800640 1 -- 172.21.15.88:0/3288392826 reap_dead start 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:15.999+0000 7f3f22800640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f0800b9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0044b0 tx=0x7f3f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab42f0f242a838d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c037070 con 0x7f3f08005210 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1c02ad80 con 0x7f3f08005210 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f20a00640 1 -- 172.21.15.88:0/3288392826 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c004080 con 0x7f3f08005210 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f28a00640 1 -- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 msgr2=0x7f3f0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f28a00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f0800b9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0044b0 tx=0x7f3f08005830 comp rx=0 tx=0).stop 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f28a00640 1 -- 172.21.15.88:0/3288392826 shutdown_connections 2024-04-26T20:02:16.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f28a00640 1 --2- 172.21.15.88:0/3288392826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f0800b9c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.000+0000 7f3f28a00640 1 -- 172.21.15.88:0/3288392826 >> 172.21.15.88:0/3288392826 conn(0x7f3f240b2d50 msgr2=0x7f3f240b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:16.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.001+0000 7f3f28a00640 1 -- 172.21.15.88:0/3288392826 shutdown_connections 2024-04-26T20:02:16.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.001+0000 7f3f28a00640 1 -- 172.21.15.88:0/3288392826 wait complete. 2024-04-26T20:02:16.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.002+0000 7f3f28a00640 1 Processor -- start 2024-04-26T20:02:16.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.002+0000 7f3f28a00640 1 -- start start 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f2414fac0 0x7f3f2414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f2409da40 0x7f3f2409dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24167c80 con 0x7f3f08005210 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24164a20 con 0x7f3f2409da40 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.003+0000 7f3f28a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3f24167e00 con 0x7f3f2414fac0 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43056/0 (socket says 172.21.15.88:43056) 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 -- 172.21.15.88:0/606353161 learned_addr learned my addr 172.21.15.88:0/606353161 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21400640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f2414fac0 0x7f3f2414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 -- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f2414fac0 msgr2=0x7f3f2414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f2414fac0 0x7f3f2414ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 -- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f2409da40 msgr2=0x7f3f2409dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f2409da40 0x7f3f2409dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 -- 172.21.15.88:0/606353161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f24150480 con 0x7f3f08005210 2024-04-26T20:02:16.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.004+0000 7f3f21e00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f14011570 tx=0x7f3f14011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.005+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14015930 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.005+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f14015ad0 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.005+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400f460 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.005+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2409e7c0 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.005+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2409ed80 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.006+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f24069540 con 0x7f3f08005210 2024-04-26T20:02:16.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.007+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f1400f600 con 0x7f3f08005210 2024-04-26T20:02:16.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.008+0000 7f3f1a000640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 0x7f3f0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:16.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.008+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f3f1408c4d0 con 0x7f3f08005210 2024-04-26T20:02:16.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.008+0000 7f3f21400640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 0x7f3f0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:16.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.009+0000 7f3f21400640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 0x7f3f0406ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f240b20c0 tx=0x7f3f10000ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:16.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.012+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f14019030 con 0x7f3f08005210 2024-04-26T20:02:16.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.252+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f24165850 con 0x7f3f08005210 2024-04-26T20:02:16.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.252+0000 7f3f1a000640 1 -- 172.21.15.88:0/606353161 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7f3f1405a7d0 con 0x7f3f08005210 2024-04-26T20:02:16.254 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 msgr2=0x7f3f0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 0x7f3f0406ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f240b20c0 tx=0x7f3f10000ed0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 msgr2=0x7f3f240d0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3f14011570 tx=0x7f3f14011a30 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 shutdown_connections 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f2409da40 0x7f3f2409dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3f0406c920 0x7f3f0406ede0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3f2414fac0 0x7f3f2414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 --2- 172.21.15.88:0/606353161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f08005210 0x7f3f240d0eb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 >> 172.21.15.88:0/606353161 conn(0x7f3f240b2d50 msgr2=0x7f3f24068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:16.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 shutdown_connections 2024-04-26T20:02:16.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:16.255+0000 7f3f28a00640 1 -- 172.21.15.88:0/606353161 wait complete. 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:02:16.276 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:16.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:16.276 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:16.334 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42455sec preferred_lft 42455sec 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:02:16.335 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ccda:b9ff:fed8:349c/64 scope link 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::8029:7dff:fe56:31fa/64 scope link 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 92:6d:d7:ca:c5:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::906d:d7ff:feca:c5f7/64 scope link 2024-04-26T20:02:16.336 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:16.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:16.337 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:02:16.416 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T20:02:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:16.420 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-26T20:02:16.472 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-26T20:02:16.472 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:02:16.472 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:02:16.472 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:02:16.472 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:02:16.473 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:02:16.473 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:02:16.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:02:16.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:02:16.473 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:02:16.474 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:02:16.582 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:02:16.677 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:16.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:16.687 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:02:16.735 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:02:16.735 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:02:16.735 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:16.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:16.735 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-26T20:02:16.817 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:02:16.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:16.818 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:16.871 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:16.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:16.871 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:16.922 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:02:17.006 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:17.060 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:02:17.061 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:02:17.062 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:17.063 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:17.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:17.064 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:02:17.114 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:02:17.115 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:02:17.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:02:17.193 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:02:17.232 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:02:17.233 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:17.234 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:17.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:17.236 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:02:17.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:2024-04-26T20:02:17.282+0000 7f2961724480 -1 init, newargv = 0x558ec3b26140 newargc=15 2024-04-26T20:02:17.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40695]: starting ceph client 2024-04-26T20:02:17.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40695]: starting fuse 2024-04-26T20:02:17.315 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:02:17.315 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-04-26T20:02:17.315 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-04-26T20:02:17.316 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-04-26T20:02:17.317 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-04-26T20:02:17.317 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:02:17.317 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:17.317 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.1")) 2024-04-26T20:02:17.317 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-04-26T20:02:17.510 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.1.40695.asok 2024-04-26T20:02:17.514 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:17.527 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40695.asok 2024-04-26T20:02:17.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:17.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40695.asok status 2024-04-26T20:02:17.755 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "1", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "40695", 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 5041, 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-04-26T20:02:17.756 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 5041 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 4137967036 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:02:17.757 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 4137967036 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.5041 192.168.0.1:0/4137967036", 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.1:0/4137967036", 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 29, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 52, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-04-26T20:02:17.758 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-26T20:02:17.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:17.770 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-04-26T20:02:17.770 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:17.823 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-04-26T20:02:17.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:17.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:17.823 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:17.902 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T20:02:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.117+0000 7ff26be00640 1 Processor -- start 2024-04-26T20:02:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.117+0000 7ff26be00640 1 -- start start 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.118+0000 7ff26be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff26c071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.118+0000 7ff26be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff26c0d1e30 0x7ff26c166b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.118+0000 7ff26be00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26c072050 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.118+0000 7ff26be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff26c070690 con 0x7ff26c0d1e30 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.118+0000 7ff26be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0703b0 con 0x7ff26c0d2590 2024-04-26T20:02:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff26a000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff26c0cf0a0 0x7ff26c0d2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:40484/0 (socket says 172.21.15.88:40484) 2024-04-26T20:02:18.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff26a000640 1 -- 172.21.15.88:0/795328539 learned_addr learned my addr 172.21.15.88:0/795328539 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff26aa00640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff26c071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff26b400640 1 --2- 172.21.15.88:0/795328539 >> v2:172.21.15.132:3300/0 conn(0x7ff26c0d1e30 0x7ff26c166b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681159718 0 0) 0x7ff254001490 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.119+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff24c003520 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1501518829 0 0) 0x7ff24c003520 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 msgr2=0x7ff26c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff26c071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 >> v2:172.21.15.132:3300/0 conn(0x7ff26c0d1e30 msgr2=0x7ff26c166b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 --2- 172.21.15.88:0/795328539 >> v2:172.21.15.132:3300/0 conn(0x7ff26c0d1e30 0x7ff26c166b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c0d0240 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7ff254001bc0 con 0x7ff26c0cf0a0 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 >> v1:172.21.15.88:6789/0 conn(0x7ff26c0cf0a0 legacy=0x7ff26c0d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff24c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff24c003f80 0x7ff24c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c004d40 0x7ff24c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c070690 con 0x7ff24c003f80 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0703b0 con 0x7ff24c004d40 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff24c00b9c0 con 0x7ff26c0d2590 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff26a000640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff24c003f80 0x7ff24c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff26aa00640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff24c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.120+0000 7ff26b400640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c004d40 0x7ff24c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.121+0000 7ff26a000640 1 -- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 msgr2=0x7ff24c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.121+0000 7ff26a000640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c0d2590 0x7ff24c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.121+0000 7ff26a000640 1 -- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c004d40 msgr2=0x7ff24c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.121+0000 7ff26a000640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c004d40 0x7ff24c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.121+0000 7ff26a000640 1 -- 172.21.15.88:0/795328539 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c003520 con 0x7ff24c003f80 2024-04-26T20:02:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.122+0000 7ff26a000640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff24c003f80 0x7ff24c004800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a9d0 tx=0x7ff24c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe5adc7a054b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.122+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25400a5c0 con 0x7ff24c003f80 2024-04-26T20:02:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.122+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25402f070 con 0x7ff24c003f80 2024-04-26T20:02:18.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.122+0000 7ff269600640 1 -- 172.21.15.88:0/795328539 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25402bde0 con 0x7ff24c003f80 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 -- 172.21.15.88:0/795328539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff24c003f80 msgr2=0x7ff24c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 --2- 172.21.15.88:0/795328539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff24c003f80 0x7ff24c004800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a9d0 tx=0x7ff24c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26b400640 1 -- 172.21.15.88:0/795328539 reap_dead start 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 -- 172.21.15.88:0/795328539 shutdown_connections 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 -- 172.21.15.88:0/795328539 >> 172.21.15.88:0/795328539 conn(0x7ff26c09ba50 msgr2=0x7ff26c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 -- 172.21.15.88:0/795328539 shutdown_connections 2024-04-26T20:02:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.123+0000 7ff26be00640 1 -- 172.21.15.88:0/795328539 wait complete. 2024-04-26T20:02:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.125+0000 7ff26be00640 1 Processor -- start 2024-04-26T20:02:18.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.125+0000 7ff26be00640 1 -- start start 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c003f80 0x7ff26c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c150070 0x7ff26c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c070060 con 0x7ff26c157f00 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0d0800 con 0x7ff24c003f80 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0d3080 con 0x7ff26c150070 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43068/0 (socket says 172.21.15.88:43068) 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 -- 172.21.15.88:0/4208606197 learned_addr learned my addr 172.21.15.88:0/4208606197 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:18.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26aa00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c003f80 0x7ff26c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26b400640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c150070 0x7ff26c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 -- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c150070 msgr2=0x7ff26c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c150070 0x7ff26c0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 -- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c003f80 msgr2=0x7ff26c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.126+0000 7ff26a000640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c003f80 0x7ff26c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff26a000640 1 -- 172.21.15.88:0/4208606197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c150560 con 0x7ff26c157f00 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff26a000640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a970 tx=0x7ff254001f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25402b9a0 con 0x7ff26c157f00 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25402bb40 con 0x7ff26c157f00 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254034470 con 0x7ff26c157f00 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c150840 con 0x7ff26c157f00 2024-04-26T20:02:18.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.127+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c150e00 con 0x7ff26c157f00 2024-04-26T20:02:18.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.128+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c069ad0 con 0x7ff26c157f00 2024-04-26T20:02:18.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.129+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff254034610 con 0x7ff26c157f00 2024-04-26T20:02:18.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.131+0000 7ff262a00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 0x7ff24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.131+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7ff2540ac260 con 0x7ff26c157f00 2024-04-26T20:02:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.131+0000 7ff26aa00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 0x7ff24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.132+0000 7ff26aa00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 0x7ff24806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009930 tx=0x7ff25c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.136+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff26c072050 con 0x7ff26c157f00 2024-04-26T20:02:18.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.389+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff26c0b1720 con 0x7ff26c157f00 2024-04-26T20:02:18.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.558+0000 7ff262a00640 1 -- 172.21.15.88:0/4208606197 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v30) ==== 136+0+0 (secure 0 0 0) 0x7ff25407a680 con 0x7ff26c157f00 2024-04-26T20:02:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 msgr2=0x7ff24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 0x7ff24806edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009930 tx=0x7ff25c008040 comp rx=0 tx=0).stop 2024-04-26T20:02:18.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 msgr2=0x7ff26c14fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff25400a970 tx=0x7ff254001f30 comp rx=0 tx=0).stop 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 shutdown_connections 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff24806c8f0 0x7ff24806edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff26c150070 0x7ff26c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff26c157f00 0x7ff26c14fb30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.560+0000 7ff26be00640 1 --2- 172.21.15.88:0/4208606197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff24c003f80 0x7ff26c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.561+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 >> 172.21.15.88:0/4208606197 conn(0x7ff26c09ba50 msgr2=0x7ff26c154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.561+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 shutdown_connections 2024-04-26T20:02:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.561+0000 7ff26be00640 1 -- 172.21.15.88:0/4208606197 wait complete. 2024-04-26T20:02:18.583 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:02:18.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.786+0000 7f497e600640 1 Processor -- start 2024-04-26T20:02:18.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.786+0000 7f497e600640 1 -- start start 2024-04-26T20:02:18.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497e600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 0x7f497815f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497e600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4978073ed0 0x7f497815fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497e600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4978073110 con 0x7f4978145570 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497e600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f49780cef40 con 0x7f4978073ed0 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497e600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f49781606b0 con 0x7f49780d1580 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4978073ed0 0x7f497815fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4978073ed0 0x7f497815fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43084/0 (socket says 172.21.15.88:43084) 2024-04-26T20:02:18.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497dc00640 1 -- 172.21.15.88:0/1789268359 learned_addr learned my addr 172.21.15.88:0/1789268359 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f497dc00640 1 -- 172.21.15.88:0/1789268359 >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 msgr2=0x7f497815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.788+0000 7f4977e00640 1 --2- 172.21.15.88:0/1789268359 >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 0x7f497815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.789+0000 7f497dc00640 1 --2- 172.21.15.88:0/1789268359 >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 0x7f497815f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.789+0000 7f497dc00640 1 -- 172.21.15.88:0/1789268359 >> v1:172.21.15.132:6789/0 conn(0x7f4978145570 legacy=0x7f49780d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.789+0000 7f497dc00640 1 -- 172.21.15.88:0/1789268359 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49780733f0 con 0x7f4978073ed0 2024-04-26T20:02:18.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.789+0000 7f4977e00640 1 --2- 172.21.15.88:0/1789268359 >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 0x7f497815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:18.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f497dc00640 1 --2- 172.21.15.88:0/1789268359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815fe80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4978072d20 tx=0x7f497002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdf6e47ee088e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f4977400640 1 -- 172.21.15.88:0/1789268359 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970036070 con 0x7f4978073ed0 2024-04-26T20:02:18.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f4977400640 1 -- 172.21.15.88:0/1789268359 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4970031600 con 0x7f4978073ed0 2024-04-26T20:02:18.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f4977400640 1 -- 172.21.15.88:0/1789268359 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49700319d0 con 0x7f4978073ed0 2024-04-26T20:02:18.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f497e600640 1 -- 172.21.15.88:0/1789268359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 msgr2=0x7f497815fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.790+0000 7f497e600640 1 --2- 172.21.15.88:0/1789268359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815fe80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4978072d20 tx=0x7f497002a660 comp rx=0 tx=0).stop 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 -- 172.21.15.88:0/1789268359 shutdown_connections 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 --2- 172.21.15.88:0/1789268359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815fe80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 --2- 172.21.15.88:0/1789268359 >> v2:172.21.15.132:3300/0 conn(0x7f49780d1580 0x7f497815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 -- 172.21.15.88:0/1789268359 >> 172.21.15.88:0/1789268359 conn(0x7f497806c740 msgr2=0x7f497806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 -- 172.21.15.88:0/1789268359 shutdown_connections 2024-04-26T20:02:18.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.791+0000 7f497e600640 1 -- 172.21.15.88:0/1789268359 wait complete. 2024-04-26T20:02:18.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.792+0000 7f497e600640 1 Processor -- start 2024-04-26T20:02:18.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.793+0000 7f497e600640 1 -- start start 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.793+0000 7f497e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.793+0000 7f497e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 0x7f49781e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.793+0000 7f497e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4978145570 0x7f49780cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49781603c0 con 0x7f4978073ed0 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f49780739b0 con 0x7f49780d1580 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4978145950 con 0x7f4978145570 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43098/0 (socket says 172.21.15.88:43098) 2024-04-26T20:02:18.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 -- 172.21.15.88:0/242117057 learned_addr learned my addr 172.21.15.88:0/242117057 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f4977e00640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 0x7f49781e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 -- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4978145570 msgr2=0x7f49780cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4978145570 0x7f49780cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 -- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 msgr2=0x7f49781e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 0x7f49781e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f497d200640 1 -- 172.21.15.88:0/242117057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49781e6d70 con 0x7f4978073ed0 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.794+0000 7f4977e00640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 0x7f49781e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:18.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f497d200640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b420 tx=0x7f496c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c011830 con 0x7f4978073ed0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496c016070 con 0x7f4978073ed0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c011c10 con 0x7f4978073ed0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49781f8210 con 0x7f4978073ed0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.795+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49781f87d0 con 0x7f4978073ed0 2024-04-26T20:02:18.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.796+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49780693c0 con 0x7f4978073ed0 2024-04-26T20:02:18.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.797+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f496c01b460 con 0x7f4978073ed0 2024-04-26T20:02:18.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.798+0000 7f4974c00640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:18.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.798+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f496c08b700 con 0x7f4978073ed0 2024-04-26T20:02:18.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.798+0000 7f4977e00640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:18.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.799+0000 7f4977e00640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 0x7f495806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49780cc0c0 tx=0x7f4964008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:18.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:18.803+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f496c020020 con 0x7f4978073ed0 2024-04-26T20:02:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.013+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f49780729e0 con 0x7f495806c8f0 2024-04-26T20:02:19.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.016+0000 7f4974c00640 1 -- 172.21.15.88:0/242117057 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f49780729e0 con 0x7f495806c8f0 2024-04-26T20:02:19.018 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:02:19.018 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:19.018 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-04-26T20:02:19.018 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:19.018 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 msgr2=0x7f495806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 0x7f495806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49780cc0c0 tx=0x7f4964008040 comp rx=0 tx=0).stop 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 msgr2=0x7f497815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b420 tx=0x7f496c00b8e0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 shutdown_connections 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4978145570 0x7f49780cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49780d1580 0x7f49781e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 --2- 172.21.15.88:0/242117057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4978073ed0 0x7f497815ba20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.018+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 >> 172.21.15.88:0/242117057 conn(0x7f497806c740 msgr2=0x7f497806e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.019+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 shutdown_connections 2024-04-26T20:02:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.019+0000 7f497e600640 1 -- 172.21.15.88:0/242117057 wait complete. 2024-04-26T20:02:19.039 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:02:19.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.232+0000 7fc78ac00640 1 Processor -- start 2024-04-26T20:02:19.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.232+0000 7fc78ac00640 1 -- start start 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc78ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc78415ee60 0x7fc7840b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc78ac00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78415d970 con 0x7fc78415c450 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc78ac00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7840b4fe0 con 0x7fc78415cf40 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc78ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc7840b52c0 con 0x7fc78415ee60 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc789800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc78415c450 0x7fc78415c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43388/0 (socket says 172.21.15.88:43388) 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc789800640 1 -- 172.21.15.88:0/739750621 learned_addr learned my addr 172.21.15.88:0/739750621 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027113893 0 0) 0x7fc78415d970 con 0x7fc78415c450 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc76c003520 con 0x7fc78415c450 2024-04-26T20:02:19.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.234+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc78415ee60 0x7fc7840b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1315583954 0 0) 0x7fc76c003520 con 0x7fc78415c450 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc78415ee60 msgr2=0x7fc7840b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc78415ee60 0x7fc7840b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 >> v1:172.21.15.132:6789/0 conn(0x7fc78415cf40 legacy=0x7fc784165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7840b55a0 con 0x7fc78415c450 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fc778001690 con 0x7fc78415c450 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 >> v1:172.21.15.88:6789/0 conn(0x7fc78415c450 legacy=0x7fc78415c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc76c003a50 0x7fc76c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc76c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc76c0053c0 0x7fc76c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840b52c0 con 0x7fc76c004800 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c00c1e0 con 0x7fc76c003a50 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c00c3e0 con 0x7fc76c0053c0 2024-04-26T20:02:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.235+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc78415ee60 0x7fc7840b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc76c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc789800640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc76c003a50 0x7fc76c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 -- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc76c0053c0 msgr2=0x7fc76c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc78a200640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc76c0053c0 0x7fc76c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc76c0053c0 0x7fc76c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 -- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc76c003a50 msgr2=0x7fc76c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc76c003a50 0x7fc76c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.236+0000 7fc788e00640 1 -- 172.21.15.88:0/739750621 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c003520 con 0x7fc76c004800 2024-04-26T20:02:19.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.237+0000 7fc788e00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc76c004c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc770006e80 tx=0x7fc76c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa35eb2ecb0ae5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.237+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770037070 con 0x7fc76c004800 2024-04-26T20:02:19.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.237+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77002aac0 con 0x7fc76c004800 2024-04-26T20:02:19.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.237+0000 7fc783e00640 1 -- 172.21.15.88:0/739750621 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f040 con 0x7fc76c004800 2024-04-26T20:02:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 -- 172.21.15.88:0/739750621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 msgr2=0x7fc76c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 --2- 172.21.15.88:0/739750621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc76c004c80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc770006e80 tx=0x7fc76c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78a200640 1 -- 172.21.15.88:0/739750621 reap_dead start 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 -- 172.21.15.88:0/739750621 shutdown_connections 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 -- 172.21.15.88:0/739750621 >> 172.21.15.88:0/739750621 conn(0x7fc7840b2900 msgr2=0x7fc7840b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 -- 172.21.15.88:0/739750621 shutdown_connections 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.238+0000 7fc78ac00640 1 -- 172.21.15.88:0/739750621 wait complete. 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.240+0000 7fc78ac00640 1 Processor -- start 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.240+0000 7fc78ac00640 1 -- start start 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc784149f50 0x7fc78414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7840723f0 0x7fc784072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840b4aa0 con 0x7fc76c004800 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840b4920 con 0x7fc7840723f0 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc78ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc7840b4c20 con 0x7fc784149f50 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc789800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc789800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52402/0 (socket says 172.21.15.88:52402) 2024-04-26T20:02:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc789800640 1 -- 172.21.15.88:0/1602466059 learned_addr learned my addr 172.21.15.88:0/1602466059 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc789800640 1 -- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc784149f50 msgr2=0x7fc78414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.241+0000 7fc789800640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc784149f50 0x7fc78414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc789800640 1 -- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7840723f0 msgr2=0x7fc784072870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc789800640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7840723f0 0x7fc784072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc789800640 1 -- 172.21.15.88:0/1602466059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78414aa10 con 0x7fc76c004800 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc789800640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc778001800 tx=0x7fc77800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800c660 con 0x7fc76c004800 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc778018070 con 0x7fc76c004800 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800ca40 con 0x7fc76c004800 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78414acf0 con 0x7fc76c004800 2024-04-26T20:02:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.242+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7841483b0 con 0x7fc76c004800 2024-04-26T20:02:19.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.243+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7841485b0 con 0x7fc76c004800 2024-04-26T20:02:19.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.244+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc77801d460 con 0x7fc76c004800 2024-04-26T20:02:19.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.246+0000 7fc781600640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 0x7fc76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.246+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fc77808bba0 con 0x7fc76c004800 2024-04-26T20:02:19.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.246+0000 7fc788e00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 0x7fc76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.247+0000 7fc788e00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 0x7fc76406edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc770034000 tx=0x7fc77002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.250+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7841485b0 con 0x7fc76c004800 2024-04-26T20:02:19.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.450+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc784147ec0 con 0x7fc76c004800 2024-04-26T20:02:19.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.450+0000 7fc781600640 1 -- 172.21.15.88:0/1602466059 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fc778059fc0 con 0x7fc76c004800 2024-04-26T20:02:19.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 msgr2=0x7fc76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 0x7fc76406edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc770034000 tx=0x7fc77002c000 comp rx=0 tx=0).stop 2024-04-26T20:02:19.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 msgr2=0x7fc7841f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc778001800 tx=0x7fc77800eb90 comp rx=0 tx=0).stop 2024-04-26T20:02:19.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 shutdown_connections 2024-04-26T20:02:19.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7840723f0 0x7fc784072870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc76406c8f0 0x7fc76406edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc784149f50 0x7fc78414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 --2- 172.21.15.88:0/1602466059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76c004800 0x7fc7841f4da0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 >> 172.21.15.88:0/1602466059 conn(0x7fc7840b2900 msgr2=0x7fc7840b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.453+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 shutdown_connections 2024-04-26T20:02:19.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.454+0000 7fc78ac00640 1 -- 172.21.15.88:0/1602466059 wait complete. 2024-04-26T20:02:19.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp86440 2024-04-26T20:02:19.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.662+0000 7f1df7e00640 1 Processor -- start 2024-04-26T20:02:19.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.662+0000 7f1df7e00640 1 -- start start 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1df006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df7e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df006eea0 con 0x7f1df00b3c30 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df006f140 con 0x7f1df0156650 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1df006f420 con 0x7f1df006c770 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df5e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f1df00b3c30 0x7f1df00b40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43400/0 (socket says 172.21.15.88:43400) 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.664+0000 7f1df5e00640 1 -- 172.21.15.88:0/1770970914 learned_addr learned my addr 172.21.15.88:0/1770970914 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:19.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1df4a00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1df006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244412254 0 0) 0x7f1df006eea0 con 0x7f1df00b3c30 2024-04-26T20:02:19.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dd8003520 con 0x7f1df00b3c30 2024-04-26T20:02:19.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1650543893 0 0) 0x7f1dd8003520 con 0x7f1df00b3c30 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 msgr2=0x7f1df006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1df006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 >> v1:172.21.15.132:6789/0 conn(0x7f1df0156650 legacy=0x7f1df015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df006f700 con 0x7f1df00b3c30 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f1de80019f0 con 0x7f1df00b3c30 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 >> v1:172.21.15.88:6789/0 conn(0x7f1df00b3c30 legacy=0x7f1df00b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1dd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8004d40 0x7f1dd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1df006f420 con 0x7f1dd80039e0 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd800bb50 con 0x7f1dd8004d40 2024-04-26T20:02:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd800df30 con 0x7f1df006c770 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.665+0000 7f1df5400640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5400640 1 -- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 msgr2=0x7f1dd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df4a00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1dd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5400640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1df006c770 0x7f1dd8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5400640 1 -- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8004d40 msgr2=0x7f1dd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5e00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8004d40 0x7f1dd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5400640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8004d40 0x7f1dd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5400640 1 -- 172.21.15.88:0/1770970914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd8003520 con 0x7f1dd80039e0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.666+0000 7f1df5e00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dd8004d40 0x7f1dd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.667+0000 7f1df5400640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001780 tx=0x7f1dd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dee944b9cdad6dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.667+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4001370 con 0x7f1dd80039e0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.667+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de40052d0 con 0x7f1dd80039e0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.667+0000 7f1defe00640 1 -- 172.21.15.88:0/1770970914 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de402ac00 con 0x7f1dd80039e0 2024-04-26T20:02:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 -- 172.21.15.88:0/1770970914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dd80039e0 msgr2=0x7f1dd8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 --2- 172.21.15.88:0/1770970914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dd80039e0 0x7f1dd8003e00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001780 tx=0x7f1dd8005830 comp rx=0 tx=0).stop 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df5e00640 1 -- 172.21.15.88:0/1770970914 reap_dead start 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 -- 172.21.15.88:0/1770970914 shutdown_connections 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 -- 172.21.15.88:0/1770970914 >> 172.21.15.88:0/1770970914 conn(0x7f1df00b2d50 msgr2=0x7f1df00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 -- 172.21.15.88:0/1770970914 shutdown_connections 2024-04-26T20:02:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.668+0000 7f1df7e00640 1 -- 172.21.15.88:0/1770970914 wait complete. 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.670+0000 7f1df7e00640 1 Processor -- start 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.670+0000 7f1df7e00640 1 -- start start 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 0x7f1df00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1df0151210 0x7f1df0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1df015c680 con 0x7f1df00ae800 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1df006fcc0 con 0x7f1df0151210 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1df0157510 con 0x7f1dd80039e0 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52422/0 (socket says 172.21.15.88:52422) 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df5e00640 1 -- 172.21.15.88:0/3564207953 learned_addr learned my addr 172.21.15.88:0/3564207953 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df5400640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 0x7f1df00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.671+0000 7f1df4a00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1df0151210 0x7f1df0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 -- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 msgr2=0x7f1df00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 0x7f1df00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 -- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1df0151210 msgr2=0x7f1df0151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1df0151210 0x7f1df0151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 -- 172.21.15.88:0/3564207953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df00af580 con 0x7f1df00ae800 2024-04-26T20:02:19.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5400640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 0x7f1df00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df5e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000f80 tx=0x7f1de800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de800bb20 con 0x7f1df00ae800 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de800bcc0 con 0x7f1df00ae800 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8007270 con 0x7f1df00ae800 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df00af860 con 0x7f1df00ae800 2024-04-26T20:02:19.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.672+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df00afe20 con 0x7f1df00ae800 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.673+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df0069650 con 0x7f1df00ae800 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.674+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1de801b020 con 0x7f1df00ae800 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.675+0000 7f1ded600640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.675+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f1de808be70 con 0x7f1df00ae800 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.675+0000 7f1df5400640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:19.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.676+0000 7f1df5400640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 0x7f1dd006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1de4032000 tx=0x7f1de402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:19.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.679+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1de805a200 con 0x7f1df00ae800 2024-04-26T20:02:19.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.910+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp86440", "target": ["mon-mgr", ""]}) -- 0x7f1df015cbc0 con 0x7f1dd006c920 2024-04-26T20:02:19.920 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:19.918+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:19.920 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:19.918+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:19.920 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:19.918+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:19.920 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:19.918+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:19.920 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:19.918+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:19.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.940+0000 7f1ded600640 1 -- 172.21.15.88:0/3564207953 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1df015cbc0 con 0x7f1dd006c920 2024-04-26T20:02:19.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 msgr2=0x7f1dd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 0x7f1dd006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1de4032000 tx=0x7f1de402e000 comp rx=0 tx=0).stop 2024-04-26T20:02:19.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 msgr2=0x7f1df00aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:19.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000f80 tx=0x7f1de800b4a0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 shutdown_connections 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df00ae800 0x7f1df00aec80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1df0151210 0x7f1df0151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1dd006c920 0x7f1dd006ede0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 --2- 172.21.15.88:0/3564207953 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1dd80039e0 0x7f1df00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 >> 172.21.15.88:0/3564207953 conn(0x7f1df00b2d50 msgr2=0x7f1df015df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 shutdown_connections 2024-04-26T20:02:19.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:19.943+0000 7f1df7e00640 1 -- 172.21.15.88:0/3564207953 wait complete. 2024-04-26T20:02:19.962 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2024-04-26T20:02:20.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.141+0000 7f4b32a00640 1 Processor -- start 2024-04-26T20:02:20.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.141+0000 7f4b32a00640 1 -- start start 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 0x7f4b2c167af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c163af0 0x7f4b2c163f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b2c06c770 0x7f4b2c06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c1614b0 con 0x7f4b2c06c770 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c164590 con 0x7f4b2c1610d0 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b32a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c164870 con 0x7f4b2c163af0 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b30a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b2c06c770 0x7f4b2c06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b30a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4b2c06c770 0x7f4b2c06cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52454/0 (socket says 172.21.15.88:52454) 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b30a00640 1 -- 172.21.15.88:0/2142844176 learned_addr learned my addr 172.21.15.88:0/2142844176 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.143+0000 7f4b2be00640 1 --2- 172.21.15.88:0/2142844176 >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 0x7f4b2c167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b2b400640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c163af0 0x7f4b2c163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b30a00640 1 -- 172.21.15.88:0/2142844176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c163af0 msgr2=0x7f4b2c163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b30a00640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c163af0 0x7f4b2c163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b30a00640 1 -- 172.21.15.88:0/2142844176 >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 msgr2=0x7f4b2c167af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b30a00640 1 --2- 172.21.15.88:0/2142844176 >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 0x7f4b2c167af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b30a00640 1 -- 172.21.15.88:0/2142844176 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1672f0 con 0x7f4b2c06c770 2024-04-26T20:02:20.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.144+0000 7f4b2be00640 1 --2- 172.21.15.88:0/2142844176 >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 0x7f4b2c167af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.145+0000 7f4b30a00640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c06c770 0x7f4b2c06cc00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c075d20 tx=0x7f4b2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b48b8d119bc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.145+0000 7f4b2aa00640 1 -- 172.21.15.88:0/2142844176 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24036070 con 0x7f4b2c06c770 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.145+0000 7f4b2aa00640 1 -- 172.21.15.88:0/2142844176 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b24031600 con 0x7f4b2c06c770 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.145+0000 7f4b2aa00640 1 -- 172.21.15.88:0/2142844176 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240319d0 con 0x7f4b2c06c770 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 -- 172.21.15.88:0/2142844176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c06c770 msgr2=0x7f4b2c06cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c06c770 0x7f4b2c06cc00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c075d20 tx=0x7f4b2402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 -- 172.21.15.88:0/2142844176 shutdown_connections 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c06c770 0x7f4b2c06cc00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 --2- 172.21.15.88:0/2142844176 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c163af0 0x7f4b2c163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 --2- 172.21.15.88:0/2142844176 >> v2:172.21.15.132:3300/0 conn(0x7f4b2c1610d0 0x7f4b2c167af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 -- 172.21.15.88:0/2142844176 >> 172.21.15.88:0/2142844176 conn(0x7f4b2c0b2d50 msgr2=0x7f4b2c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 -- 172.21.15.88:0/2142844176 shutdown_connections 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.146+0000 7f4b32a00640 1 -- 172.21.15.88:0/2142844176 wait complete. 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.148+0000 7f4b32a00640 1 Processor -- start 2024-04-26T20:02:20.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.148+0000 7f4b32a00640 1 -- start start 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c06c770 0x7f4b2c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2c163af0 0x7f4b2c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c0b6040 con 0x7f4b2c1610d0 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c0b5ec0 con 0x7f4b2c163af0 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b32a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c0add80 con 0x7f4b2c06c770 2024-04-26T20:02:20.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52464/0 (socket says 172.21.15.88:52464) 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 -- 172.21.15.88:0/3144296745 learned_addr learned my addr 172.21.15.88:0/3144296745 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2be00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c06c770 0x7f4b2c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 -- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c06c770 msgr2=0x7f4b2c06ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c06c770 0x7f4b2c06ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b30a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2c163af0 0x7f4b2c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.149+0000 7f4b2b400640 1 -- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2c163af0 msgr2=0x7f4b2c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b2b400640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2c163af0 0x7f4b2c0cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b2b400640 1 -- 172.21.15.88:0/3144296745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c15d400 con 0x7f4b2c1610d0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b2b400640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c7f0 tx=0x7f4b1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18015070 con 0x7f4b2c1610d0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18004220 con 0x7f4b2c1610d0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c06ee80 con 0x7f4b2c1610d0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18007620 con 0x7f4b2c1610d0 2024-04-26T20:02:20.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.150+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c06f440 con 0x7f4b2c1610d0 2024-04-26T20:02:20.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.151+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c069650 con 0x7f4b2c1610d0 2024-04-26T20:02:20.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.152+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b180079d0 con 0x7f4b2c1610d0 2024-04-26T20:02:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.153+0000 7f4b23e00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 0x7f4b0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.153+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f4b1808ba70 con 0x7f4b2c1610d0 2024-04-26T20:02:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.153+0000 7f4b2be00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 0x7f4b0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.154+0000 7f4b2be00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 0x7f4b0c06ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c008850 tx=0x7f4b1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.158+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b18059e90 con 0x7f4b2c1610d0 2024-04-26T20:02:20.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.386+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f4b2c0af530 con 0x7f4b0c06c920 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.392+0000 7f4b23e00640 1 -- 172.21.15.88:0/3144296745 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+530 (secure 0 0 0) 0x7f4b2c0af530 con 0x7f4b0c06c920 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "mon_addrs": [ 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.88:6789", 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.132:6789", 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.132:6790" 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "pending_subvolume_deletions": 0, 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "pools": { 2024-04-26T20:02:20.394 INFO:teuthology.orchestra.run.smithi088.stdout: "data": [ 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "avail": 367087058944, 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs_data", 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "used": 0 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": [ 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "avail": 367087058944, 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs_metadata", 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: "used": 65536 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-26T20:02:20.395 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:20.396 INFO:teuthology.orchestra.run.smithi088.stdout: "used_size": 0 2024-04-26T20:02:20.396 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.394+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 msgr2=0x7f4b0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.394+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 0x7f4b0c06ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c008850 tx=0x7f4b1c008040 comp rx=0 tx=0).stop 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.394+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 msgr2=0x7f4b2c152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.394+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c7f0 tx=0x7f4b1800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 shutdown_connections 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4b2c163af0 0x7f4b2c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b2c1610d0 0x7f4b2c152f10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4b0c06c920 0x7f4b0c06ede0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 --2- 172.21.15.88:0/3144296745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4b2c06c770 0x7f4b2c06ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 >> 172.21.15.88:0/3144296745 conn(0x7f4b2c0b2d50 msgr2=0x7f4b2c0b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:20.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 shutdown_connections 2024-04-26T20:02:20.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.395+0000 7f4b32a00640 1 -- 172.21.15.88:0/3144296745 wait complete. 2024-04-26T20:02:20.414 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:02:20.414 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:02:20.516 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:02:20.636 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:02:20.636 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:02:20.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:20.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:20.693 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:20.693 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:20.744 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:20.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.945+0000 7fae59400640 1 Processor -- start 2024-04-26T20:02:20.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.946+0000 7fae59400640 1 -- start start 2024-04-26T20:02:20.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae59400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae540d0e70 0x7fae540d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae59400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 0x7fae540b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae59400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae54146a60 con 0x7fae54146680 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae59400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fae5406e970 con 0x7fae540d0e70 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae59400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fae540b60a0 con 0x7fae540b3b20 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.947+0000 7fae53400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae540d0e70 0x7fae540d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae540d0e70 0x7fae540d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52480/0 (socket says 172.21.15.88:52480) 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 -- 172.21.15.88:0/2372128566 learned_addr learned my addr 172.21.15.88:0/2372128566 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae58a00640 1 --2- 172.21.15.88:0/2372128566 >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 0x7fae540b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 -- 172.21.15.88:0/2372128566 >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 msgr2=0x7fae540b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 --2- 172.21.15.88:0/2372128566 >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 0x7fae540b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 -- 172.21.15.88:0/2372128566 >> v1:172.21.15.132:6789/0 conn(0x7fae54146680 legacy=0x7fae540b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae53400640 1 -- 172.21.15.88:0/2372128566 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5406dd60 con 0x7fae540d0e70 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.948+0000 7fae58a00640 1 --2- 172.21.15.88:0/2372128566 >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 0x7fae540b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.949+0000 7fae53400640 1 --2- 172.21.15.88:0/2372128566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540d0e70 0x7fae540d12b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fae5406f370 tx=0x7fae3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cd19a26167f8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.949+0000 7fae52a00640 1 -- 172.21.15.88:0/2372128566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c037070 con 0x7fae540d0e70 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.949+0000 7fae52a00640 1 -- 172.21.15.88:0/2372128566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c032440 con 0x7fae540d0e70 2024-04-26T20:02:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.949+0000 7fae52a00640 1 -- 172.21.15.88:0/2372128566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c032810 con 0x7fae540d0e70 2024-04-26T20:02:20.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 -- 172.21.15.88:0/2372128566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540d0e70 msgr2=0x7fae540d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 --2- 172.21.15.88:0/2372128566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540d0e70 0x7fae540d12b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fae5406f370 tx=0x7fae3c02a230 comp rx=0 tx=0).stop 2024-04-26T20:02:20.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 -- 172.21.15.88:0/2372128566 shutdown_connections 2024-04-26T20:02:20.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 --2- 172.21.15.88:0/2372128566 >> v2:172.21.15.132:3300/0 conn(0x7fae540b3b20 0x7fae540b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 --2- 172.21.15.88:0/2372128566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540d0e70 0x7fae540d12b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.950+0000 7fae59400640 1 -- 172.21.15.88:0/2372128566 >> 172.21.15.88:0/2372128566 conn(0x7fae5406c740 msgr2=0x7fae5406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:20.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.951+0000 7fae59400640 1 -- 172.21.15.88:0/2372128566 shutdown_connections 2024-04-26T20:02:20.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.951+0000 7fae59400640 1 -- 172.21.15.88:0/2372128566 wait complete. 2024-04-26T20:02:20.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.952+0000 7fae59400640 1 Processor -- start 2024-04-26T20:02:20.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.952+0000 7fae59400640 1 -- start start 2024-04-26T20:02:20.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae540d0e70 0x7fae5413a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae54146680 0x7fae5413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae5406e320 con 0x7fae540b3b20 2024-04-26T20:02:20.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fae540d17f0 con 0x7fae54146680 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.953+0000 7fae59400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fae5406e600 con 0x7fae540d0e70 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52490/0 (socket says 172.21.15.88:52490) 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 -- 172.21.15.88:0/2220220893 learned_addr learned my addr 172.21.15.88:0/2220220893 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae540d0e70 0x7fae5413a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae58a00640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae54146680 0x7fae5413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 -- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae540d0e70 msgr2=0x7fae5413a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae540d0e70 0x7fae5413a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 -- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae54146680 msgr2=0x7fae5413a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae54146680 0x7fae5413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 -- 172.21.15.88:0/2220220893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae540cbec0 con 0x7fae540b3b20 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.954+0000 7fae53e00640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fae4400ba50 tx=0x7fae4400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.955+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44019070 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.955+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae4400cc20 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.955+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44014680 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.955+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae540cc1a0 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.955+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae540cc760 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.956+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae540693c0 con 0x7fae540b3b20 2024-04-26T20:02:20.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.957+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae4401c070 con 0x7fae540b3b20 2024-04-26T20:02:20.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.958+0000 7fae4be00640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 0x7fae3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:20.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.958+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fae4408b7f0 con 0x7fae540b3b20 2024-04-26T20:02:20.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.959+0000 7fae53400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 0x7fae3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:20.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.959+0000 7fae53400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 0x7fae3006ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae3c034000 tx=0x7fae3c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:20.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:20.963+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae44059570 con 0x7fae540b3b20 2024-04-26T20:02:21.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.163+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae54141c20 con 0x7fae540b3b20 2024-04-26T20:02:21.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.166+0000 7fae4be00640 1 -- 172.21.15.88:0/2220220893 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7fae4405d870 con 0x7fae540b3b20 2024-04-26T20:02:21.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:21.169 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 msgr2=0x7fae3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 0x7fae3006ede0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae3c034000 tx=0x7fae3c02e000 comp rx=0 tx=0).stop 2024-04-26T20:02:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 msgr2=0x7fae5415c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fae4400ba50 tx=0x7fae4400bf10 comp rx=0 tx=0).stop 2024-04-26T20:02:21.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 shutdown_connections 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae54146680 0x7fae5413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae3006c920 0x7fae3006ede0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae540d0e70 0x7fae5413a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 --2- 172.21.15.88:0/2220220893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae540b3b20 0x7fae5415c530 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 >> 172.21.15.88:0/2220220893 conn(0x7fae5406c740 msgr2=0x7fae5406cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.169+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 shutdown_connections 2024-04-26T20:02:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.170+0000 7fae59400640 1 -- 172.21.15.88:0/2220220893 wait complete. 2024-04-26T20:02:21.190 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:59.312423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:00.563428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:02:21.191 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:02:21.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.398+0000 7fd136c00640 1 Processor -- start 2024-04-26T20:02:21.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.398+0000 7fd136c00640 1 -- start start 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 0x7fd13015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd13006c770 0x7fd13006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 0x7fd1300735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd13006eea0 con 0x7fd13006c770 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd13006f140 con 0x7fd130156650 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd136c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd13006f420 con 0x7fd130073120 2024-04-26T20:02:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd127400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd13006c770 0x7fd13006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd127400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd13006c770 0x7fd13006cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52494/0 (socket says 172.21.15.88:52494) 2024-04-26T20:02:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.400+0000 7fd127400640 1 -- 172.21.15.88:0/649770211 learned_addr learned my addr 172.21.15.88:0/649770211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd134c00640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 0x7fd1300735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd12fe00640 1 --2- 172.21.15.88:0/649770211 >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 0x7fd13015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd127400640 1 -- 172.21.15.88:0/649770211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 msgr2=0x7fd1300735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd127400640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 0x7fd1300735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd127400640 1 -- 172.21.15.88:0/649770211 >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 msgr2=0x7fd13015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd127400640 1 --2- 172.21.15.88:0/649770211 >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 0x7fd13015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd127400640 1 -- 172.21.15.88:0/649770211 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13006f700 con 0x7fd13006c770 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd134c00640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 0x7fd1300735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.401+0000 7fd12fe00640 1 --2- 172.21.15.88:0/649770211 >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 0x7fd13015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.402+0000 7fd127400640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd13006cbf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd130075cd0 tx=0x7fd11802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23eaf9801f994724 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.402+0000 7fd12f400640 1 -- 172.21.15.88:0/649770211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118037070 con 0x7fd13006c770 2024-04-26T20:02:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.402+0000 7fd12f400640 1 -- 172.21.15.88:0/649770211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118032440 con 0x7fd13006c770 2024-04-26T20:02:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.402+0000 7fd12f400640 1 -- 172.21.15.88:0/649770211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118032810 con 0x7fd13006c770 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 -- 172.21.15.88:0/649770211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 msgr2=0x7fd13006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd13006cbf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd130075cd0 tx=0x7fd11802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 -- 172.21.15.88:0/649770211 shutdown_connections 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130073120 0x7fd1300735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 --2- 172.21.15.88:0/649770211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd13006cbf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 --2- 172.21.15.88:0/649770211 >> v2:172.21.15.132:3300/0 conn(0x7fd130156650 0x7fd13015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 -- 172.21.15.88:0/649770211 >> 172.21.15.88:0/649770211 conn(0x7fd1300b2d50 msgr2=0x7fd1300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 -- 172.21.15.88:0/649770211 shutdown_connections 2024-04-26T20:02:21.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.403+0000 7fd136c00640 1 -- 172.21.15.88:0/649770211 wait complete. 2024-04-26T20:02:21.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.405+0000 7fd136c00640 1 Processor -- start 2024-04-26T20:02:21.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.405+0000 7fd136c00640 1 -- start start 2024-04-26T20:02:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd130073120 0x7fd1300cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 0x7fd130151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd13015d320 con 0x7fd13006c770 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd13006fcc0 con 0x7fd130073120 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd136c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd13015c680 con 0x7fd130156650 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.406+0000 7fd12fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52498/0 (socket says 172.21.15.88:52498) 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 -- 172.21.15.88:0/2890840427 learned_addr learned my addr 172.21.15.88:0/2890840427 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd134c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 0x7fd130151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd127400640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd130073120 0x7fd1300cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 -- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 msgr2=0x7fd130151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 0x7fd130151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 -- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd130073120 msgr2=0x7fd1300cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd130073120 0x7fd1300cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 -- 172.21.15.88:0/2890840427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1301521b0 con 0x7fd13006c770 2024-04-26T20:02:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd134c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 0x7fd130151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12fe00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c7f0 tx=0x7fd12000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.407+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120015070 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.408+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1200079d0 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.408+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1300cef90 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.408+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120007da0 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.408+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1300cd640 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.409+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd130069650 con 0x7fd13006c770 2024-04-26T20:02:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.410+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd120018070 con 0x7fd13006c770 2024-04-26T20:02:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.411+0000 7fd12cc00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 0x7fd10c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.411+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fd12008bab0 con 0x7fd13006c770 2024-04-26T20:02:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.411+0000 7fd127400640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 0x7fd10c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.412+0000 7fd127400640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 0x7fd10c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd118034000 tx=0x7fd11802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.415+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd120059ed0 con 0x7fd13006c770 2024-04-26T20:02:21.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.599+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd120055e40 con 0x7fd13006c770 2024-04-26T20:02:21.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.615+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd13015cbc0 con 0x7fd13006c770 2024-04-26T20:02:21.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.616+0000 7fd12cc00640 1 -- 172.21.15.88:0/2890840427 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7fd120018360 con 0x7fd13006c770 2024-04-26T20:02:21.618 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:02:21.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.618+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 msgr2=0x7fd10c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.618+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 0x7fd10c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd118034000 tx=0x7fd11802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.618+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 msgr2=0x7fd1300ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.618+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd12000c7f0 tx=0x7fd12000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 shutdown_connections 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd10c06c8f0 0x7fd10c06edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd130156650 0x7fd130151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd130073120 0x7fd1300cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 --2- 172.21.15.88:0/2890840427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd13006c770 0x7fd1300ce920 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 >> 172.21.15.88:0/2890840427 conn(0x7fd1300b2d50 msgr2=0x7fd1300b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 shutdown_connections 2024-04-26T20:02:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.619+0000 7fd136c00640 1 -- 172.21.15.88:0/2890840427 wait complete. 2024-04-26T20:02:21.638 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:02:21.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.827+0000 7fb2c3200640 1 Processor -- start 2024-04-26T20:02:21.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.827+0000 7fb2c3200640 1 -- start start 2024-04-26T20:02:21.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c3200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb2c415f7f0 0x7fb2c40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415fbd0 0x7fb2c40d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c3200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c40d0170 con 0x7fb2c4162cf0 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c3200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c40d0450 con 0x7fb2c415f7f0 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c40d0730 con 0x7fb2c415fbd0 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c2800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb2c4162cf0 0x7fb2c4163180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43406/0 (socket says 172.21.15.88:43406) 2024-04-26T20:02:21.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.829+0000 7fb2c2800640 1 -- 172.21.15.88:0/1298356563 learned_addr learned my addr 172.21.15.88:0/1298356563 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415fbd0 0x7fb2c40d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> v2:172.21.15.132:3300/0 conn(0x7fb2c415f7f0 0x7fb2c40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671767398 0 0) 0x7fb2c40d0170 con 0x7fb2c4162cf0 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2a8003520 con 0x7fb2c4162cf0 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 252595077 0 0) 0x7fb2a8003520 con 0x7fb2c4162cf0 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415fbd0 msgr2=0x7fb2c40d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415fbd0 0x7fb2c40d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 >> v2:172.21.15.132:3300/0 conn(0x7fb2c415f7f0 msgr2=0x7fb2c40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 --2- 172.21.15.88:0/1298356563 >> v2:172.21.15.132:3300/0 conn(0x7fb2c415f7f0 0x7fb2c40cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c40d0a10 con 0x7fb2c4162cf0 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb2bc001670 con 0x7fb2c4162cf0 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 >> v1:172.21.15.88:6789/0 conn(0x7fb2c4162cf0 legacy=0x7fb2c4163180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2a8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2a8004800 0x7fb2a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2a8005210 0x7fb2a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40d0450 con 0x7fb2a8003a20 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40d0730 con 0x7fb2a8004800 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a800c140 con 0x7fb2a8005210 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> v2:172.21.15.132:3300/0 conn(0x7fb2c415f7f0 0x7fb2c40cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.830+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415fbd0 0x7fb2c40d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2a8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2a8005210 msgr2=0x7fb2a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2a8005210 0x7fb2a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2a8004800 msgr2=0x7fb2a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2a8004800 0x7fb2a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2a8004800 0x7fb2a8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.831+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1298356563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a8003520 con 0x7fb2a8003a20 2024-04-26T20:02:21.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.832+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2a8003e40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40073e0 tx=0x7fb2a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1684f77c8d475eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.832+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4009c70 con 0x7fb2a8003a20 2024-04-26T20:02:21.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.832+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b402c680 con 0x7fb2a8003a20 2024-04-26T20:02:21.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.832+0000 7fb2c0a00640 1 -- 172.21.15.88:0/1298356563 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b402ca30 con 0x7fb2a8003a20 2024-04-26T20:02:21.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 -- 172.21.15.88:0/1298356563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 msgr2=0x7fb2a8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1298356563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2a8003e40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40073e0 tx=0x7fb2a8005830 comp rx=0 tx=0).stop 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c2800640 1 -- 172.21.15.88:0/1298356563 reap_dead start 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 -- 172.21.15.88:0/1298356563 shutdown_connections 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 -- 172.21.15.88:0/1298356563 >> 172.21.15.88:0/1298356563 conn(0x7fb2c40d4f80 msgr2=0x7fb2c40d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 -- 172.21.15.88:0/1298356563 shutdown_connections 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.833+0000 7fb2c3200640 1 -- 172.21.15.88:0/1298356563 wait complete. 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.835+0000 7fb2c3200640 1 Processor -- start 2024-04-26T20:02:21.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.835+0000 7fb2c3200640 1 -- start start 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415b4f0 0x7fb2c415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2c415d510 0x7fb2c41f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c415f430 con 0x7fb2a8003a20 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4160050 con 0x7fb2c415d510 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c3200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c415f5b0 con 0x7fb2c415b4f0 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52512/0 (socket says 172.21.15.88:52512) 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1809974373 learned_addr learned my addr 172.21.15.88:0/1809974373 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:21.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.836+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415b4f0 0x7fb2c415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c2800640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2c415d510 0x7fb2c41f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415b4f0 msgr2=0x7fb2c415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415b4f0 0x7fb2c415cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2c415d510 msgr2=0x7fb2c41f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2c415d510 0x7fb2c41f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 -- 172.21.15.88:0/1809974373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c41f5e80 con 0x7fb2a8003a20 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c1e00640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4007500 tx=0x7fb2b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4009c70 con 0x7fb2a8003a20 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b402c480 con 0x7fb2a8003a20 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b402dc20 con 0x7fb2a8003a20 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c41f5ba0 con 0x7fb2a8003a20 2024-04-26T20:02:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.837+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c40cc460 con 0x7fb2a8003a20 2024-04-26T20:02:21.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.838+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c40693c0 con 0x7fb2a8003a20 2024-04-26T20:02:21.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.839+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb2b4049030 con 0x7fb2a8003a20 2024-04-26T20:02:21.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.840+0000 7fb2ba000640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 0x7fb2a006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:21.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.840+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b40ad080 con 0x7fb2a8003a20 2024-04-26T20:02:21.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.841+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 0x7fb2a006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:21.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.841+0000 7fb2c1400640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 0x7fb2a006ee70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:21.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:21.845+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2b4050060 con 0x7fb2a8003a20 2024-04-26T20:02:22.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.034+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2c408ea90 con 0x7fb2a8003a20 2024-04-26T20:02:22.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.035+0000 7fb2ba000640 1 -- 172.21.15.88:0/1809974373 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7fb2b403c040 con 0x7fb2a8003a20 2024-04-26T20:02:22.037 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 msgr2=0x7fb2a006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 0x7fb2a006ee70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 msgr2=0x7fb2c40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4007500 tx=0x7fb2b4004ab0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 shutdown_connections 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2c415d510 0x7fb2c41f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb2a006c9b0 0x7fb2a006ee70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb2c415b4f0 0x7fb2c415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 --2- 172.21.15.88:0/1809974373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2a8003a20 0x7fb2c40ccc60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.037+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 >> 172.21.15.88:0/1809974373 conn(0x7fb2c40d4f80 msgr2=0x7fb2c41608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.038+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 shutdown_connections 2024-04-26T20:02:22.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.038+0000 7fb2c3200640 1 -- 172.21.15.88:0/1809974373 wait complete. 2024-04-26T20:02:22.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:02:22.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.248+0000 7f26e5200640 1 Processor -- start 2024-04-26T20:02:22.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.249+0000 7f26e5200640 1 -- start start 2024-04-26T20:02:22.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26e5200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26e00d3110 0x7f26e0162410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26e5200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f26e00b6210 0x7f26e0162950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26e5200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e00b6710 con 0x7f26e01632e0 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26e5200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f26e00d3600 con 0x7f26e00d3110 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26e5200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f26e00d38e0 con 0x7f26e00b6210 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.250+0000 7f26dea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26e00d3110 0x7f26e0162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f26e00d3110 0x7f26e0162410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52518/0 (socket says 172.21.15.88:52518) 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 -- 172.21.15.88:0/3492161996 learned_addr learned my addr 172.21.15.88:0/3492161996 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dfe00640 1 --2- 172.21.15.88:0/3492161996 >> v2:172.21.15.132:3300/0 conn(0x7f26e00b6210 0x7f26e0162950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 -- 172.21.15.88:0/3492161996 >> v2:172.21.15.132:3300/0 conn(0x7f26e00b6210 msgr2=0x7f26e0162950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:22.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 --2- 172.21.15.88:0/3492161996 >> v2:172.21.15.132:3300/0 conn(0x7f26e00b6210 0x7f26e0162950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 -- 172.21.15.88:0/3492161996 >> v1:172.21.15.132:6789/0 conn(0x7f26e01632e0 legacy=0x7f26e01636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:22.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.251+0000 7f26dea00640 1 -- 172.21.15.88:0/3492161996 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00d3bc0 con 0x7f26e00d3110 2024-04-26T20:02:22.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.252+0000 7f26dea00640 1 --2- 172.21.15.88:0/3492161996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00d3110 0x7f26e0162410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f26e00d4a30 tx=0x7f26c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ee638f09de5f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.252+0000 7f26de000640 1 -- 172.21.15.88:0/3492161996 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8036070 con 0x7f26e00d3110 2024-04-26T20:02:22.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.252+0000 7f26de000640 1 -- 172.21.15.88:0/3492161996 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c8031600 con 0x7f26e00d3110 2024-04-26T20:02:22.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.252+0000 7f26de000640 1 -- 172.21.15.88:0/3492161996 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80319d0 con 0x7f26e00d3110 2024-04-26T20:02:22.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 -- 172.21.15.88:0/3492161996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00d3110 msgr2=0x7f26e0162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 --2- 172.21.15.88:0/3492161996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00d3110 0x7f26e0162410 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f26e00d4a30 tx=0x7f26c802a660 comp rx=0 tx=0).stop 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 -- 172.21.15.88:0/3492161996 shutdown_connections 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 --2- 172.21.15.88:0/3492161996 >> v2:172.21.15.132:3300/0 conn(0x7f26e00b6210 0x7f26e0162950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 --2- 172.21.15.88:0/3492161996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00d3110 0x7f26e0162410 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.253+0000 7f26e5200640 1 -- 172.21.15.88:0/3492161996 >> 172.21.15.88:0/3492161996 conn(0x7f26e009e2f0 msgr2=0x7f26e00d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.254+0000 7f26e5200640 1 -- 172.21.15.88:0/3492161996 shutdown_connections 2024-04-26T20:02:22.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.254+0000 7f26e5200640 1 -- 172.21.15.88:0/3492161996 wait complete. 2024-04-26T20:02:22.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.255+0000 7f26e5200640 1 Processor -- start 2024-04-26T20:02:22.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.255+0000 7f26e5200640 1 -- start start 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e00d3110 0x7f26e00ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e01632e0 0x7f26e015f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f26e00b2250 con 0x7f26e00b6210 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0163dd0 con 0x7f26e00d3110 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.256+0000 7f26e5200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f26e00b23d0 con 0x7f26e01632e0 2024-04-26T20:02:22.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52526/0 (socket says 172.21.15.88:52526) 2024-04-26T20:02:22.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 -- 172.21.15.88:0/2985405301 learned_addr learned my addr 172.21.15.88:0/2985405301 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 -- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e01632e0 msgr2=0x7f26e015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26dfe00640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e01632e0 0x7f26e015f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26dea00640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e00d3110 0x7f26e00ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e01632e0 0x7f26e015f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 -- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e00d3110 msgr2=0x7f26e00ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e00d3110 0x7f26e00ba200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 -- 172.21.15.88:0/2985405301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00bac50 con 0x7f26e00b6210 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.257+0000 7f26df400640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f26d000b3f0 tx=0x7f26d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.258+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0015020 con 0x7f26e00b6210 2024-04-26T20:02:22.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.258+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d0011830 con 0x7f26e00b6210 2024-04-26T20:02:22.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.258+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0011c20 con 0x7f26e00b6210 2024-04-26T20:02:22.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.258+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e01f53a0 con 0x7f26e00b6210 2024-04-26T20:02:22.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.258+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01f4d40 con 0x7f26e00b6210 2024-04-26T20:02:22.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.259+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e00693c0 con 0x7f26e00b6210 2024-04-26T20:02:22.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.260+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f26d0018020 con 0x7f26e00b6210 2024-04-26T20:02:22.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.261+0000 7f26d7400640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 0x7f26bc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.261+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f26d008b8c0 con 0x7f26e00b6210 2024-04-26T20:02:22.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.261+0000 7f26dea00640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 0x7f26bc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.262+0000 7f26dea00640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 0x7f26bc06ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8033000 tx=0x7f26c8009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.267+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26d001f030 con 0x7f26e00b6210 2024-04-26T20:02:22.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.464+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26e009e8b0 con 0x7f26e00b6210 2024-04-26T20:02:22.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.465+0000 7f26d7400640 1 -- 172.21.15.88:0/2985405301 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f26d0059c00 con 0x7f26e00b6210 2024-04-26T20:02:22.467 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 msgr2=0x7f26bc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 0x7f26bc06ee40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f26c8033000 tx=0x7f26c8009000 comp rx=0 tx=0).stop 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 msgr2=0x7f26e0161060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f26d000b3f0 tx=0x7f26d000b8b0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 shutdown_connections 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f26bc06c980 0x7f26bc06ee40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f26e01632e0 0x7f26e015f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26e00d3110 0x7f26e00ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 --2- 172.21.15.88:0/2985405301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26e00b6210 0x7f26e0161060 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 >> 172.21.15.88:0/2985405301 conn(0x7f26e009e2f0 msgr2=0x7f26e00d5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 shutdown_connections 2024-04-26T20:02:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.468+0000 7f26e5200640 1 -- 172.21.15.88:0/2985405301 wait complete. 2024-04-26T20:02:22.489 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:02:22.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.691+0000 7f9bfc800640 1 Processor -- start 2024-04-26T20:02:22.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.691+0000 7f9bfc800640 1 -- start start 2024-04-26T20:02:22.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.692+0000 7f9bfc800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f9bf415cb70 0x7f9bf406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bfc800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf40ae850 con 0x7f9bf415cfd0 2024-04-26T20:02:22.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bfc800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf40aeb30 con 0x7f9bf406c770 2024-04-26T20:02:22.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bfc800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40af460 con 0x7f9bf415cb70 2024-04-26T20:02:22.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bfa800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f9bf415cfd0 0x7f9bf406f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43422/0 (socket says 172.21.15.88:43422) 2024-04-26T20:02:22.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bfa800640 1 -- 172.21.15.88:0/4066734733 learned_addr learned my addr 172.21.15.88:0/4066734733 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bf9400640 1 --2- 172.21.15.88:0/4066734733 >> v2:172.21.15.132:3300/0 conn(0x7f9bf415cb70 0x7f9bf406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182979081 0 0) 0x7f9bf40ae850 con 0x7f9bf415cfd0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.693+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be0005730 con 0x7f9bf415cfd0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 673306028 0 0) 0x7f9be0005730 con 0x7f9bf415cfd0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 >> v2:172.21.15.132:3300/0 conn(0x7f9bf415cb70 msgr2=0x7f9bf406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 --2- 172.21.15.88:0/4066734733 >> v2:172.21.15.132:3300/0 conn(0x7f9bf415cb70 0x7f9bf406ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 >> v1:172.21.15.132:6789/0 conn(0x7f9bf406c770 legacy=0x7f9bf406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf40aee10 con 0x7f9bf415cfd0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f9bf0001670 con 0x7f9bf415cfd0 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 >> v1:172.21.15.88:6789/0 conn(0x7f9bf415cfd0 legacy=0x7f9bf406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9be0005bc0 0x7f9be0006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 0x7f9be0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be00073d0 0x7f9be000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40af460 con 0x7f9be0006a10 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9be000e2a0 con 0x7f9be0005bc0 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9be000e4a0 con 0x7f9be00073d0 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.694+0000 7f9bf9400640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 0x7f9be0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bf9400640 1 -- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be00073d0 msgr2=0x7f9be000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bf9400640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be00073d0 0x7f9be000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bf9400640 1 -- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9be0005bc0 msgr2=0x7f9be0006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bf9400640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9be0005bc0 0x7f9be0006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bf9400640 1 -- 172.21.15.88:0/4066734733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be0005730 con 0x7f9be0006a10 2024-04-26T20:02:22.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.695+0000 7f9bfa800640 1 -- 172.21.15.88:0/4066734733 reap_dead start 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bf9400640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 0x7f9be0006e90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009ac0 tx=0x7f9be0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941f2bf7c485012b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4036070 con 0x7f9be0006a10 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4031400 con 0x7f9be0006a10 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bf8a00640 1 -- 172.21.15.88:0/4066734733 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4009690 con 0x7f9be0006a10 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bfc800640 1 -- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 msgr2=0x7f9be0006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.696+0000 7f9bfc800640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 0x7f9be0006e90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009ac0 tx=0x7f9be0007a40 comp rx=0 tx=0).stop 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.697+0000 7f9bfc800640 1 -- 172.21.15.88:0/4066734733 shutdown_connections 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.697+0000 7f9bfc800640 1 --2- 172.21.15.88:0/4066734733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9be0006a10 0x7f9be0006e90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.697+0000 7f9bfc800640 1 -- 172.21.15.88:0/4066734733 >> 172.21.15.88:0/4066734733 conn(0x7f9bf40b2d50 msgr2=0x7f9bf40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:22.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.697+0000 7f9bfc800640 1 -- 172.21.15.88:0/4066734733 shutdown_connections 2024-04-26T20:02:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.697+0000 7f9bfc800640 1 -- 172.21.15.88:0/4066734733 wait complete. 2024-04-26T20:02:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.698+0000 7f9bfc800640 1 Processor -- start 2024-04-26T20:02:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.698+0000 7f9bfc800640 1 -- start start 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.699+0000 7f9bfc800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 0x7f9bf41fa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.699+0000 7f9bfc800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bfc800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf41e6830 0x7f9bf41e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bfc800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf415c780 con 0x7f9bf4158af0 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bfc800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf415c600 con 0x7f9bf41e6830 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bfc800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf40af8f0 con 0x7f9be0006a10 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52552/0 (socket says 172.21.15.88:52552) 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 -- 172.21.15.88:0/258798134 learned_addr learned my addr 172.21.15.88:0/258798134 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9e00640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 0x7f9bf41fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 -- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 msgr2=0x7f9bf41fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bfa800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf41e6830 0x7f9bf41e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 0x7f9bf41fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 -- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf41e6830 msgr2=0x7f9bf41e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf41e6830 0x7f9bf41e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9400640 1 -- 172.21.15.88:0/258798134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf41fd2e0 con 0x7f9bf4158af0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.700+0000 7f9bf9e00640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 0x7f9bf41fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bf9400640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009be0 tx=0x7f9be4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4036050 con 0x7f9bf4158af0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be4037040 con 0x7f9bf4158af0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40044e0 con 0x7f9bf4158af0 2024-04-26T20:02:22.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41fd5c0 con 0x7f9bf4158af0 2024-04-26T20:02:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.701+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf41fdb80 con 0x7f9bf4158af0 2024-04-26T20:02:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.702+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf40d1a40 con 0x7f9bf4158af0 2024-04-26T20:02:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.703+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9be4044020 con 0x7f9bf4158af0 2024-04-26T20:02:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.704+0000 7f9bea000640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 0x7f9bd806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.704+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f9be40abef0 con 0x7f9bf4158af0 2024-04-26T20:02:22.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.704+0000 7f9bf9e00640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 0x7f9bd806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:22.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.705+0000 7f9bf9e00640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 0x7f9bd806ee70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007fd0 tx=0x7f9bec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:22.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.709+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9bf40d1a40 con 0x7f9bf4158af0 2024-04-26T20:02:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.924+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9bf41f7230 con 0x7f9bf4158af0 2024-04-26T20:02:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.925+0000 7f9bea000640 1 -- 172.21.15.88:0/258798134 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f9bf41f7230 con 0x7f9bf4158af0 2024-04-26T20:02:22.927 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 msgr2=0x7f9bd806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 0x7f9bd806ee70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007fd0 tx=0x7f9bec009040 comp rx=0 tx=0).stop 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 msgr2=0x7f9bf409ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009be0 tx=0x7f9be4004730 comp rx=0 tx=0).stop 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 shutdown_connections 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf41e6830 0x7f9bf41e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf4158af0 0x7f9bf409ba50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f9bd806c9b0 0x7f9bd806ee70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 --2- 172.21.15.88:0/258798134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f9be0006a10 0x7f9bf41fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:22.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 >> 172.21.15.88:0/258798134 conn(0x7f9bf40b2d50 msgr2=0x7f9bf40b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:22.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 shutdown_connections 2024-04-26T20:02:22.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:22.928+0000 7f9bfc800640 1 -- 172.21.15.88:0/258798134 wait complete. 2024-04-26T20:02:22.948 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-04-26T20:02:22.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:22.949 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:23.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[50562]: fuse finished with error 0 and tester_r 0 2024-04-26T20:02:23.028 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:02:23.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:23.029 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:02:23.184 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:02:23.185 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:02:23.185 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:23.244 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:02:23.244 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:23.299 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:23.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.501+0000 7f93d7200640 1 Processor -- start 2024-04-26T20:02:23.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.501+0000 7f93d7200640 1 -- start start 2024-04-26T20:02:23.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d81523a0 0x7f93d815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93d8152780 0x7f93d80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 0x7f93d80b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f93d806dbc0 con 0x7f93d8152780 2024-04-26T20:02:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f93d806de10 con 0x7f93d80723e0 2024-04-26T20:02:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93d7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f93d806e0f0 con 0x7f93d81523a0 2024-04-26T20:02:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93cd400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93d8152780 0x7f93d80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93cd400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f93d8152780 0x7f93d80d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52558/0 (socket says 172.21.15.88:52558) 2024-04-26T20:02:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.502+0000 7f93cd400640 1 -- 172.21.15.88:0/4100173859 learned_addr learned my addr 172.21.15.88:0/4100173859 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93d5e00640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d81523a0 0x7f93d815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93d6800640 1 --2- 172.21.15.88:0/4100173859 >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 0x7f93d80b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93cd400640 1 -- 172.21.15.88:0/4100173859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d81523a0 msgr2=0x7f93d815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:23.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93cd400640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d81523a0 0x7f93d815c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93cd400640 1 -- 172.21.15.88:0/4100173859 >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 msgr2=0x7f93d80b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:23.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93cd400640 1 --2- 172.21.15.88:0/4100173859 >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 0x7f93d80b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93cd400640 1 -- 172.21.15.88:0/4100173859 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d806e3d0 con 0x7f93d8152780 2024-04-26T20:02:23.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.503+0000 7f93d6800640 1 --2- 172.21.15.88:0/4100173859 >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 0x7f93d80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:23.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.504+0000 7f93cd400640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d80d0300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f93d806fe70 tx=0x7f93c002b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d70a9e809842b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.504+0000 7f93d5400640 1 -- 172.21.15.88:0/4100173859 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0038070 con 0x7f93d8152780 2024-04-26T20:02:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.504+0000 7f93d5400640 1 -- 172.21.15.88:0/4100173859 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c0033640 con 0x7f93d8152780 2024-04-26T20:02:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.504+0000 7f93d5400640 1 -- 172.21.15.88:0/4100173859 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0033a10 con 0x7f93d8152780 2024-04-26T20:02:23.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 -- 172.21.15.88:0/4100173859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 msgr2=0x7f93d80d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d80d0300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f93d806fe70 tx=0x7f93c002b2f0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 -- 172.21.15.88:0/4100173859 shutdown_connections 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 --2- 172.21.15.88:0/4100173859 >> v2:172.21.15.132:3300/0 conn(0x7f93d80723e0 0x7f93d80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d80d0300 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 --2- 172.21.15.88:0/4100173859 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d81523a0 0x7f93d815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 -- 172.21.15.88:0/4100173859 >> 172.21.15.88:0/4100173859 conn(0x7f93d806c740 msgr2=0x7f93d806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 -- 172.21.15.88:0/4100173859 shutdown_connections 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.505+0000 7f93d7200640 1 -- 172.21.15.88:0/4100173859 wait complete. 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.507+0000 7f93d7200640 1 Processor -- start 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.507+0000 7f93d7200640 1 -- start start 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d80723e0 0x7f93d815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 0x7f93d815e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8072990 con 0x7f93d8152780 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8072810 con 0x7f93d81523a0 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80b4ab0 con 0x7f93d80723e0 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52560/0 (socket says 172.21.15.88:52560) 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 -- 172.21.15.88:0/1493924030 learned_addr learned my addr 172.21.15.88:0/1493924030 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93cd400640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 0x7f93d815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d5e00640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d80723e0 0x7f93d815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 -- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d80723e0 msgr2=0x7f93d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d80723e0 0x7f93d815a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 -- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 msgr2=0x7f93d815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 0x7f93d815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93d6800640 1 -- 172.21.15.88:0/1493924030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80cd9a0 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.508+0000 7f93cd400640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 0x7f93d815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93d6800640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f93d000cef0 tx=0x7f93d000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d001a070 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d000d840 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d000dc30 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d80cdc80 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.509+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d80ce240 con 0x7f93d8152780 2024-04-26T20:02:23.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.510+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d80693c0 con 0x7f93d8152780 2024-04-26T20:02:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.511+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f93d001d070 con 0x7f93d8152780 2024-04-26T20:02:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.512+0000 7f93cea00640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 0x7f93b406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.512+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f93d008c3d0 con 0x7f93d8152780 2024-04-26T20:02:23.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.512+0000 7f93d5e00640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 0x7f93b406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.513+0000 7f93d5e00640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 0x7f93b406ee70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006fd0 tx=0x7f93c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.516+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93d005a7a0 con 0x7f93d8152780 2024-04-26T20:02:23.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.718+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93d80721c0 con 0x7f93d8152780 2024-04-26T20:02:23.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.720+0000 7f93cea00640 1 -- 172.21.15.88:0/1493924030 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7f93d005a0a0 con 0x7f93d8152780 2024-04-26T20:02:23.722 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:23.723 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 msgr2=0x7f93b406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 0x7f93b406ee70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006fd0 tx=0x7f93c8008040 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 msgr2=0x7f93d815e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f93d000cef0 tx=0x7f93d000a710 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 shutdown_connections 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93d8152780 0x7f93d815e840 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d81523a0 0x7f93d815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f93b406c9b0 0x7f93b406ee70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 --2- 172.21.15.88:0/1493924030 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93d80723e0 0x7f93d815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 >> 172.21.15.88:0/1493924030 conn(0x7f93d806c740 msgr2=0x7f93d80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 shutdown_connections 2024-04-26T20:02:23.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.723+0000 7f93d7200640 1 -- 172.21.15.88:0/1493924030 wait complete. 2024-04-26T20:02:23.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:59.312423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:00.563428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:02:23.744 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:02:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.950+0000 7f3ce1000640 1 Processor -- start 2024-04-26T20:02:23.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.950+0000 7f3ce1000640 1 -- start start 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3ce1000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3cdc06ce80 0x7f3cdc0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 0x7f3cdc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3ce1000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cdc0af800 con 0x7f3cdc15d100 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3ce1000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3cdc0ae340 con 0x7f3cdc06ce80 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3ce1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc0afaa0 con 0x7f3cdc0b5ca0 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3cd3e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3cdc06ce80 0x7f3cdc0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3cd9e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3cdc15d100 0x7f3cdc06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43428/0 (socket says 172.21.15.88:43428) 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.951+0000 7f3cd9e00640 1 -- 172.21.15.88:0/3133388002 learned_addr learned my addr 172.21.15.88:0/3133388002 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cda800640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 0x7f3cdc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cd3e00640 1 -- 172.21.15.88:0/3133388002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 msgr2=0x7f3cdc0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cd3e00640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 0x7f3cdc0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cd3e00640 1 -- 172.21.15.88:0/3133388002 >> v1:172.21.15.88:6789/0 conn(0x7f3cdc15d100 legacy=0x7f3cdc06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cd9400640 1 -- 172.21.15.88:0/3133388002 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3045628166 0 0) 0x7f3cdc0af800 con 0x7f3cdc15d100 2024-04-26T20:02:23.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cd3e00640 1 -- 172.21.15.88:0/3133388002 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc0afd80 con 0x7f3cdc06ce80 2024-04-26T20:02:23.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.952+0000 7f3cda800640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 0x7f3cdc0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:23.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.953+0000 7f3cd3e00640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc06ce80 0x7f3cdc0b3c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40097a0 tx=0x7f3cc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4880e570ee15e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.953+0000 7f3cd9400640 1 -- 172.21.15.88:0/3133388002 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4037070 con 0x7f3cdc06ce80 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.953+0000 7f3cd9400640 1 -- 172.21.15.88:0/3133388002 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4032440 con 0x7f3cdc06ce80 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.953+0000 7f3cd9400640 1 -- 172.21.15.88:0/3133388002 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4032810 con 0x7f3cdc06ce80 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 -- 172.21.15.88:0/3133388002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc06ce80 msgr2=0x7f3cdc0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc06ce80 0x7f3cdc0b3c30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40097a0 tx=0x7f3cc402a180 comp rx=0 tx=0).stop 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 -- 172.21.15.88:0/3133388002 shutdown_connections 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc0b5ca0 0x7f3cdc0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 --2- 172.21.15.88:0/3133388002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc06ce80 0x7f3cdc0b3c30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 -- 172.21.15.88:0/3133388002 >> 172.21.15.88:0/3133388002 conn(0x7f3cdc0b2d50 msgr2=0x7f3cdc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 -- 172.21.15.88:0/3133388002 shutdown_connections 2024-04-26T20:02:23.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.954+0000 7f3ce1000640 1 -- 172.21.15.88:0/3133388002 wait complete. 2024-04-26T20:02:23.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.956+0000 7f3ce1000640 1 Processor -- start 2024-04-26T20:02:23.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.956+0000 7f3ce1000640 1 -- start start 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 0x7f3cdc150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cdc0b5ca0 0x7f3cdc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc09ee50 con 0x7f3cdc15d100 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc0b0340 con 0x7f3cdc0b5ca0 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3ce1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3cdc160b70 con 0x7f3cdc06ce80 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3cda800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3cda800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52580/0 (socket says 172.21.15.88:52580) 2024-04-26T20:02:23.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.957+0000 7f3cda800640 1 -- 172.21.15.88:0/600499752 learned_addr learned my addr 172.21.15.88:0/600499752 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cd9e00640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 0x7f3cdc150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cd3e00640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cdc0b5ca0 0x7f3cdc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 -- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 msgr2=0x7f3cdc150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 0x7f3cdc150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 -- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cdc0b5ca0 msgr2=0x7f3cdc15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cdc0b5ca0 0x7f3cdc15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 -- 172.21.15.88:0/600499752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cdc15a6f0 con 0x7f3cdc15d100 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cd9e00640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 0x7f3cdc150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cda800640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400cf20 tx=0x7f3cd400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.958+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd401a070 con 0x7f3cdc15d100 2024-04-26T20:02:23.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.959+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd400d840 con 0x7f3cdc15d100 2024-04-26T20:02:23.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.959+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc15a9d0 con 0x7f3cdc15d100 2024-04-26T20:02:23.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.959+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400dc30 con 0x7f3cdc15d100 2024-04-26T20:02:23.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.959+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc15af90 con 0x7f3cdc15d100 2024-04-26T20:02:23.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.960+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cdc069650 con 0x7f3cdc15d100 2024-04-26T20:02:23.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.961+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3cd401d070 con 0x7f3cdc15d100 2024-04-26T20:02:23.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.962+0000 7f3cd2000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 0x7f3cb806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:23.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.962+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f3cd408c4f0 con 0x7f3cdc15d100 2024-04-26T20:02:23.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.962+0000 7f3cd9e00640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 0x7f3cb806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:23.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.963+0000 7f3cd9e00640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 0x7f3cb806ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000f60 tx=0x7f3ccc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:23.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:23.966+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cd405a830 con 0x7f3cdc15d100 2024-04-26T20:02:24.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.166+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cdc15c440 con 0x7f3cdc15d100 2024-04-26T20:02:24.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.167+0000 7f3cd2000640 1 -- 172.21.15.88:0/600499752 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f3cd405a130 con 0x7f3cdc15d100 2024-04-26T20:02:24.169 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 msgr2=0x7f3cb806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 0x7f3cb806ee40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc000f60 tx=0x7f3ccc009040 comp rx=0 tx=0).stop 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 msgr2=0x7f3cdc154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400cf20 tx=0x7f3cd400a7d0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 shutdown_connections 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cdc15d100 0x7f3cdc154080 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3cdc0b5ca0 0x7f3cdc15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3cb806c980 0x7f3cb806ee40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 --2- 172.21.15.88:0/600499752 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3cdc06ce80 0x7f3cdc150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 >> 172.21.15.88:0/600499752 conn(0x7f3cdc0b2d50 msgr2=0x7f3cdc0b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:24.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.169+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 shutdown_connections 2024-04-26T20:02:24.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.170+0000 7f3ce1000640 1 -- 172.21.15.88:0/600499752 wait complete. 2024-04-26T20:02:24.193 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:02:24.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.386+0000 7f4ef3a00640 1 Processor -- start 2024-04-26T20:02:24.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.387+0000 7f4ef3a00640 1 -- start start 2024-04-26T20:02:24.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef3a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4eec09c160 0x7f4eec0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef3a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0d2f10 con 0x7f4eec0b5710 2024-04-26T20:02:24.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef3a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eec0d1dd0 con 0x7f4eec162ab0 2024-04-26T20:02:24.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef3a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4eec0d23e0 con 0x7f4eec09c160 2024-04-26T20:02:24.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef1a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4eec0b5710 0x7f4eec0cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43434/0 (socket says 172.21.15.88:43434) 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.388+0000 7f4ef1a00640 1 -- 172.21.15.88:0/411716886 learned_addr learned my addr 172.21.15.88:0/411716886 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eebe00640 1 --2- 172.21.15.88:0/411716886 >> v2:172.21.15.132:3300/0 conn(0x7f4eec09c160 0x7f4eec0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434616608 0 0) 0x7f4eec0d2f10 con 0x7f4eec0b5710 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ed0003520 con 0x7f4eec0b5710 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1824109627 0 0) 0x7f4ed0003520 con 0x7f4eec0b5710 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 >> v2:172.21.15.132:3300/0 conn(0x7f4eec09c160 msgr2=0x7f4eec0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 --2- 172.21.15.88:0/411716886 >> v2:172.21.15.132:3300/0 conn(0x7f4eec09c160 0x7f4eec0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 >> v1:172.21.15.132:6789/0 conn(0x7f4eec162ab0 legacy=0x7f4eec09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec162e90 con 0x7f4eec0b5710 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f4ee4001670 con 0x7f4eec0b5710 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.389+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 >> v1:172.21.15.88:6789/0 conn(0x7f4eec0b5710 legacy=0x7f4eec0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0003a00 0x7f4ed0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ed0005210 0x7f4ed000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0d23e0 con 0x7f4ed0004800 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed000c160 con 0x7f4ed0005210 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed000c360 con 0x7f4ed0003a00 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4ef1000640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0003a00 0x7f4ed0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 -- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0003a00 msgr2=0x7f4ed0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0003a00 0x7f4ed0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 -- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ed0005210 msgr2=0x7f4ed000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ed0005210 0x7f4ed000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4eebe00640 1 -- 172.21.15.88:0/411716886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0003520 con 0x7f4ed0004800 2024-04-26T20:02:24.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4ef1a00640 1 -- 172.21.15.88:0/411716886 reap_dead start 2024-04-26T20:02:24.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.390+0000 7f4ef1000640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0003a00 0x7f4ed0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.391+0000 7f4eebe00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009a90 tx=0x7f4ed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bad35e182d9dde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8036070 con 0x7f4ed0004800 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed8031400 con 0x7f4ed0004800 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4eeb400640 1 -- 172.21.15.88:0/411716886 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8009690 con 0x7f4ed0004800 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 -- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 msgr2=0x7f4ed0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009a90 tx=0x7f4ed0005830 comp rx=0 tx=0).stop 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 -- 172.21.15.88:0/411716886 shutdown_connections 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/411716886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed0004800 0x7f4ed0004c80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 -- 172.21.15.88:0/411716886 >> 172.21.15.88:0/411716886 conn(0x7f4eec0b4bf0 msgr2=0x7f4eec0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 -- 172.21.15.88:0/411716886 shutdown_connections 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.392+0000 7f4ef3a00640 1 -- 172.21.15.88:0/411716886 wait complete. 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.394+0000 7f4ef3a00640 1 Processor -- start 2024-04-26T20:02:24.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.394+0000 7f4ef3a00640 1 -- start start 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0004800 0x7f4eec153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec0cbf70 0x7f4eec15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0b25a0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4eec0d2070 con 0x7f4eec0cbf70 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4eec0cbd20 con 0x7f4ed0004800 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.395+0000 7f4ef1a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52610/0 (socket says 172.21.15.88:52610) 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 -- 172.21.15.88:0/1547672688 learned_addr learned my addr 172.21.15.88:0/1547672688 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1000640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0004800 0x7f4eec153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4eebe00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec0cbf70 0x7f4eec15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 -- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0004800 msgr2=0x7f4eec153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0004800 0x7f4eec153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 -- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec0cbf70 msgr2=0x7f4eec15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec0cbf70 0x7f4eec15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 -- 172.21.15.88:0/1547672688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eec0ca5b0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.396+0000 7f4ef1a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c450 tx=0x7f4ee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.397+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004bf0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.397+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ee4004db0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.397+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007270 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.397+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eec0ca890 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.397+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eec0cae50 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.398+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eec0cb230 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.399+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4ee4007410 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.400+0000 7f4ee8c00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 0x7f4ecc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.400+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f4ee408b1d0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.400+0000 7f4ef1000640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 0x7f4ecc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.401+0000 7f4ef1000640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 0x7f4ecc06ee70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007fd0 tx=0x7f4edc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.404+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4eec0cb230 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.601+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4eec0c74d0 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.606+0000 7f4ee8c00640 1 -- 172.21.15.88:0/1547672688 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7f4ee400e280 con 0x7f4eec0cc3f0 2024-04-26T20:02:24.608 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:02:24.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.609+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 msgr2=0x7f4ecc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.609+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 0x7f4ecc06ee70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4edc007fd0 tx=0x7f4edc009040 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.609+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 msgr2=0x7f4eec15e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.609+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400c450 tx=0x7f4ee400c910 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 shutdown_connections 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec0cc3f0 0x7f4eec15e590 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec0cbf70 0x7f4eec15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4ecc06c9b0 0x7f4ecc06ee70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 --2- 172.21.15.88:0/1547672688 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ed0004800 0x7f4eec153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 >> 172.21.15.88:0/1547672688 conn(0x7f4eec0b4bf0 msgr2=0x7f4eec06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 shutdown_connections 2024-04-26T20:02:24.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.610+0000 7f4ef3a00640 1 -- 172.21.15.88:0/1547672688 wait complete. 2024-04-26T20:02:24.630 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:02:24.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.817+0000 7f45b4c00640 1 Processor -- start 2024-04-26T20:02:24.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.817+0000 7f45b4c00640 1 -- start start 2024-04-26T20:02:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.818+0000 7f45b4c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f45b0152430 0x7f45b00b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.818+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.818+0000 7f45b4c00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b006e130 con 0x7f45b0072320 2024-04-26T20:02:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.818+0000 7f45b4c00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f45b0073090 con 0x7f45b0152430 2024-04-26T20:02:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.818+0000 7f45b4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f45b0073370 con 0x7f45b0152810 2024-04-26T20:02:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.819+0000 7f45ade00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f45b0152430 0x7f45b00b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.819+0000 7f45a5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.819+0000 7f45ae800640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f45b0072320 0x7f45b0072780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:44378/0 (socket says 172.21.15.88:44378) 2024-04-26T20:02:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.819+0000 7f45a5400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:33554/0 (socket says 172.21.15.88:33554) 2024-04-26T20:02:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.819+0000 7f45ae800640 1 -- 172.21.15.88:0/3288168955 learned_addr learned my addr 172.21.15.88:0/3288168955 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.820+0000 7f45ade00640 1 -- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 msgr2=0x7f45b00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.820+0000 7f45ade00640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.820+0000 7f45ade00640 1 -- 172.21.15.88:0/3288168955 >> v1:172.21.15.132:6789/0 conn(0x7f45b0072320 legacy=0x7f45b0072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.820+0000 7f45ade00640 1 -- 172.21.15.88:0/3288168955 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b0073650 con 0x7f45b0152430 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.820+0000 7f45a5400640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:02:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.821+0000 7f45ade00640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b00b4500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45a00097a0 tx=0x7f45a002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8b76aa720c5d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.822+0000 7f45ad400640 1 -- 172.21.15.88:0/3288168955 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0037070 con 0x7f45b0152430 2024-04-26T20:02:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.822+0000 7f45ad400640 1 -- 172.21.15.88:0/3288168955 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a0032440 con 0x7f45b0152430 2024-04-26T20:02:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.822+0000 7f45ad400640 1 -- 172.21.15.88:0/3288168955 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0032810 con 0x7f45b0152430 2024-04-26T20:02:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.822+0000 7f45b4c00640 1 -- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 msgr2=0x7f45b00b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.822+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b00b4500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f45a00097a0 tx=0x7f45a002a230 comp rx=0 tx=0).stop 2024-04-26T20:02:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 -- 172.21.15.88:0/3288168955 shutdown_connections 2024-04-26T20:02:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3288168955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b00b4500 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 -- 172.21.15.88:0/3288168955 >> 172.21.15.88:0/3288168955 conn(0x7f45b00b2d50 msgr2=0x7f45b00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 -- 172.21.15.88:0/3288168955 shutdown_connections 2024-04-26T20:02:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.823+0000 7f45b4c00640 1 -- 172.21.15.88:0/3288168955 wait complete. 2024-04-26T20:02:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.824+0000 7f45b4c00640 1 Processor -- start 2024-04-26T20:02:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 -- start start 2024-04-26T20:02:24.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b014c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45b006c8f0 con 0x7f45b0072320 2024-04-26T20:02:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f45b006c770 con 0x7f45b0152430 2024-04-26T20:02:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.825+0000 7f45b4c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f45b006ca70 con 0x7f45b0152810 2024-04-26T20:02:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52616/0 (socket says 172.21.15.88:52616) 2024-04-26T20:02:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 -- 172.21.15.88:0/3403322559 learned_addr learned my addr 172.21.15.88:0/3403322559 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 -- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 msgr2=0x7f45b01508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 -- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 msgr2=0x7f45b014c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45a5400640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b014c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b014c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.826+0000 7f45ade00640 1 -- 172.21.15.88:0/3403322559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b014a2b0 con 0x7f45b0072320 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45ade00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f45a0000c00 tx=0x7f45a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0041070 con 0x7f45b0072320 2024-04-26T20:02:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a003c870 con 0x7f45b0072320 2024-04-26T20:02:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b014a590 con 0x7f45b0072320 2024-04-26T20:02:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0037040 con 0x7f45b0072320 2024-04-26T20:02:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.827+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b014ab50 con 0x7f45b0072320 2024-04-26T20:02:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.828+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b009eef0 con 0x7f45b0072320 2024-04-26T20:02:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.829+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f45a0044070 con 0x7f45b0072320 2024-04-26T20:02:24.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.830+0000 7f45a6a00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 0x7f458c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:24.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.830+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f45a00ac6f0 con 0x7f45b0072320 2024-04-26T20:02:24.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.830+0000 7f45a5400640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 0x7f458c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:24.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.831+0000 7f45a5400640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 0x7f458c06ee40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45b01695e0 tx=0x7f4598008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:24.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:24.834+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45b009eef0 con 0x7f45b0072320 2024-04-26T20:02:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.036+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45b0069650 con 0x7f45b0072320 2024-04-26T20:02:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.037+0000 7f45a6a00640 1 -- 172.21.15.88:0/3403322559 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f45a007a9a0 con 0x7f45b0072320 2024-04-26T20:02:25.039 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:02:25.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.039+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 msgr2=0x7f458c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.039+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 0x7f458c06ee40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45b01695e0 tx=0x7f4598008040 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.039+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 msgr2=0x7f45b014b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.039+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f45a0000c00 tx=0x7f45a00046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 shutdown_connections 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f458c06c980 0x7f458c06ee40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f45b0152810 0x7f45b01508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45b0152430 0x7f45b014c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 --2- 172.21.15.88:0/3403322559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b0072320 0x7f45b014b350 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 >> 172.21.15.88:0/3403322559 conn(0x7f45b00b2d50 msgr2=0x7f45b015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 shutdown_connections 2024-04-26T20:02:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.040+0000 7f45b4c00640 1 -- 172.21.15.88:0/3403322559 wait complete. 2024-04-26T20:02:25.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:02:25.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.261+0000 7fb67de00640 1 Processor -- start 2024-04-26T20:02:25.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.262+0000 7fb67de00640 1 -- start start 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb67de00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb67806ce80 0x7fb6780add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb67de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb6780b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb67de00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6780af9a0 con 0x7fb67815cdf0 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb67de00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb6780ae2c0 con 0x7fb67806ce80 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb67de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb6780b05b0 con 0x7fb6780cd650 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb66fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb67806ce80 0x7fb6780add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb676a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb67815cdf0 0x7fb67806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43450/0 (socket says 172.21.15.88:43450) 2024-04-26T20:02:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.263+0000 7fb676a00640 1 -- 172.21.15.88:0/1832066283 learned_addr learned my addr 172.21.15.88:0/1832066283 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb66fe00640 1 -- 172.21.15.88:0/1832066283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 msgr2=0x7fb6780b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb677400640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb6780b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb66fe00640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb6780b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb66fe00640 1 -- 172.21.15.88:0/1832066283 >> v1:172.21.15.88:6789/0 conn(0x7fb67815cdf0 legacy=0x7fb67806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb66fe00640 1 -- 172.21.15.88:0/1832066283 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6780afc80 con 0x7fb67806ce80 2024-04-26T20:02:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb677400640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb6780b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.264+0000 7fb676000640 1 -- 172.21.15.88:0/1832066283 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602607489 0 0) 0x7fb6780af9a0 con 0x7fb67815cdf0 2024-04-26T20:02:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.265+0000 7fb66fe00640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67806ce80 0x7fb6780add80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097a0 tx=0x7fb66002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ffd1ca0650bd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.265+0000 7fb676000640 1 -- 172.21.15.88:0/1832066283 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660037070 con 0x7fb67806ce80 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.265+0000 7fb676000640 1 -- 172.21.15.88:0/1832066283 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb660032440 con 0x7fb67806ce80 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.265+0000 7fb676000640 1 -- 172.21.15.88:0/1832066283 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660032810 con 0x7fb67806ce80 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 -- 172.21.15.88:0/1832066283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67806ce80 msgr2=0x7fb6780add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67806ce80 0x7fb6780add80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097a0 tx=0x7fb66002a180 comp rx=0 tx=0).stop 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 -- 172.21.15.88:0/1832066283 shutdown_connections 2024-04-26T20:02:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb6780b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 --2- 172.21.15.88:0/1832066283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67806ce80 0x7fb6780add80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 -- 172.21.15.88:0/1832066283 >> 172.21.15.88:0/1832066283 conn(0x7fb6780b2d50 msgr2=0x7fb6780b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 -- 172.21.15.88:0/1832066283 shutdown_connections 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.266+0000 7fb67de00640 1 -- 172.21.15.88:0/1832066283 wait complete. 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.268+0000 7fb67de00640 1 Processor -- start 2024-04-26T20:02:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.268+0000 7fb67de00640 1 -- start start 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb67806ce80 0x7fb6780cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb678156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb67815d1d0 con 0x7fb67815cdf0 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780b0240 con 0x7fb67806ce80 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb67de00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb67815d350 con 0x7fb6780cd650 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb677400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb677400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52624/0 (socket says 172.21.15.88:52624) 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.269+0000 7fb677400640 1 -- 172.21.15.88:0/3157654415 learned_addr learned my addr 172.21.15.88:0/3157654415 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb66fe00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb678156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 -- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 msgr2=0x7fb678156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb678156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 -- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb67806ce80 msgr2=0x7fb6780cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb67806ce80 0x7fb6780cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 -- 172.21.15.88:0/3157654415 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6781f2a50 con 0x7fb67815cdf0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb677400640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb67000cef0 tx=0x7fb67000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67001a070 con 0x7fb67815cdf0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67000d860 con 0x7fb67815cdf0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67000dc50 con 0x7fb67815cdf0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6781603b0 con 0x7fb67815cdf0 2024-04-26T20:02:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.270+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6781600d0 con 0x7fb67815cdf0 2024-04-26T20:02:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.272+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb678069540 con 0x7fb67815cdf0 2024-04-26T20:02:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.272+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb67001d070 con 0x7fb67815cdf0 2024-04-26T20:02:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.273+0000 7fb66f400640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.273+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fb67008c3e0 con 0x7fb67815cdf0 2024-04-26T20:02:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.274+0000 7fb676a00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.274+0000 7fb676a00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 0x7fb65406ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:25.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.278+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb67005a970 con 0x7fb67815cdf0 2024-04-26T20:02:25.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.493+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb678156cd0 con 0x7fb67815cdf0 2024-04-26T20:02:25.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.494+0000 7fb66f400640 1 -- 172.21.15.88:0/3157654415 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7fb67005a270 con 0x7fb67815cdf0 2024-04-26T20:02:25.496 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 msgr2=0x7fb65406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 0x7fb65406ee40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb668007fd0 tx=0x7fb668007920 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 msgr2=0x7fb67815f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb67000cef0 tx=0x7fb67000a710 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 shutdown_connections 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb67815cdf0 0x7fb67815f580 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6780cd650 0x7fb678156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 --2- 172.21.15.88:0/3157654415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb67806ce80 0x7fb6780cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:25.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 >> 172.21.15.88:0/3157654415 conn(0x7fb6780b2d50 msgr2=0x7fb67806c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:25.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 shutdown_connections 2024-04-26T20:02:25.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.497+0000 7fb67de00640 1 -- 172.21.15.88:0/3157654415 wait complete. 2024-04-26T20:02:25.518 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi132.front.sepia.ceph.com... 2024-04-26T20:02:25.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:25.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:25.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[40695]: fuse finished with error 0 and tester_r 0 2024-04-26T20:02:25.609 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:02:25.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:25.610 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:02:25.678 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:02:25.678 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:25.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:25.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:02:25.736 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:02:25.736 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:02:25.736 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:25.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:02:25.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:26.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:25.999+0000 7fa98ec00640 1 Processor -- start 2024-04-26T20:02:26.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.000+0000 7fa98ec00640 1 -- start start 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa98ec00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa98806c770 0x7fa98806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa98ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 0x7fa988073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa98ec00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98806d090 con 0x7fa98815cb70 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa98ec00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa9880af460 con 0x7fa98806c770 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa98ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa9880ae850 con 0x7fa98815cfd0 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa987e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa98806c770 0x7fa98806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa97f400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa98815cb70 0x7fa988073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43462/0 (socket says 172.21.15.88:43462) 2024-04-26T20:02:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.001+0000 7fa97f400640 1 -- 172.21.15.88:0/1209026353 learned_addr learned my addr 172.21.15.88:0/1209026353 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:26.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa987e00640 1 -- 172.21.15.88:0/1209026353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 msgr2=0x7fa988073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa98cc00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 0x7fa988073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa987e00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 0x7fa988073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa987400640 1 -- 172.21.15.88:0/1209026353 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128957187 0 0) 0x7fa98806d090 con 0x7fa98815cb70 2024-04-26T20:02:26.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa987e00640 1 -- 172.21.15.88:0/1209026353 >> v1:172.21.15.88:6789/0 conn(0x7fa98815cb70 legacy=0x7fa988073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa987e00640 1 -- 172.21.15.88:0/1209026353 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9880aeb30 con 0x7fa98806c770 2024-04-26T20:02:26.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.002+0000 7fa98cc00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 0x7fa988073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:26.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.003+0000 7fa987e00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98806c770 0x7fa98806cb50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097a0 tx=0x7fa97802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a90e9c795b07e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.003+0000 7fa987400640 1 -- 172.21.15.88:0/1209026353 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978037070 con 0x7fa98806c770 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.003+0000 7fa987400640 1 -- 172.21.15.88:0/1209026353 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978032440 con 0x7fa98806c770 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.003+0000 7fa987400640 1 -- 172.21.15.88:0/1209026353 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978032810 con 0x7fa98806c770 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 -- 172.21.15.88:0/1209026353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98806c770 msgr2=0x7fa98806cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98806c770 0x7fa98806cb50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097a0 tx=0x7fa97802a180 comp rx=0 tx=0).stop 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 -- 172.21.15.88:0/1209026353 shutdown_connections 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98815cfd0 0x7fa988073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1209026353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98806c770 0x7fa98806cb50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 -- 172.21.15.88:0/1209026353 >> 172.21.15.88:0/1209026353 conn(0x7fa9880b2d50 msgr2=0x7fa9880b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:26.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 -- 172.21.15.88:0/1209026353 shutdown_connections 2024-04-26T20:02:26.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.004+0000 7fa98ec00640 1 -- 172.21.15.88:0/1209026353 wait complete. 2024-04-26T20:02:26.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.006+0000 7fa98ec00640 1 Processor -- start 2024-04-26T20:02:26.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.006+0000 7fa98ec00640 1 -- start start 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 0x7fa988164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98815cb70 0x7fa98815fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa98815c600 con 0x7fa98815cfd0 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa9880af0f0 con 0x7fa98815cb70 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98ec00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa98815c780 con 0x7fa98806c770 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52644/0 (socket says 172.21.15.88:52644) 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa98cc00640 1 -- 172.21.15.88:0/1528178838 learned_addr learned my addr 172.21.15.88:0/1528178838 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa987e00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 0x7fa988164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.007+0000 7fa97f400640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98815cb70 0x7fa98815fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 -- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 msgr2=0x7fa988164ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 0x7fa988164ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 -- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98815cb70 msgr2=0x7fa98815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98815cb70 0x7fa98815fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 -- 172.21.15.88:0/1528178838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9881684a0 con 0x7fa98815cfd0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa987e00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 0x7fa988164ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98cc00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cd80 tx=0x7fa98000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000f680 con 0x7fa98815cfd0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98000f840 con 0x7fa98815cfd0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000fc30 con 0x7fa98815cfd0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa988168780 con 0x7fa98815cfd0 2024-04-26T20:02:26.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.008+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa988168d40 con 0x7fa98815cfd0 2024-04-26T20:02:26.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.010+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988069540 con 0x7fa98815cfd0 2024-04-26T20:02:26.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.010+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa98000e3e0 con 0x7fa98815cfd0 2024-04-26T20:02:26.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.011+0000 7fa984c00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 0x7fa96406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.011+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fa98008cba0 con 0x7fa98815cfd0 2024-04-26T20:02:26.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.012+0000 7fa987e00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 0x7fa96406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.012+0000 7fa987e00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 0x7fa96406ee40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa978034000 tx=0x7fa97802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.016+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa980014030 con 0x7fa98815cfd0 2024-04-26T20:02:26.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.263+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9880cd430 con 0x7fa98815cfd0 2024-04-26T20:02:26.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.264+0000 7fa984c00640 1 -- 172.21.15.88:0/1528178838 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5181 (secure 0 0 0) 0x7fa98005add0 con 0x7fa98815cfd0 2024-04-26T20:02:26.266 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:26.266 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:26.266 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.266+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 msgr2=0x7fa96406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.266+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 0x7fa96406ee40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa978034000 tx=0x7fa97802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.266+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 msgr2=0x7fa988153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.266+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cd80 tx=0x7fa98000a650 comp rx=0 tx=0).stop 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.266+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 shutdown_connections 2024-04-26T20:02:26.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa98815cfd0 0x7fa988153150 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa98815cb70 0x7fa98815fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa96406c980 0x7fa96406ee40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 --2- 172.21.15.88:0/1528178838 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa98806c770 0x7fa988164ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 >> 172.21.15.88:0/1528178838 conn(0x7fa9880b2d50 msgr2=0x7fa9880b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 shutdown_connections 2024-04-26T20:02:26.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.267+0000 7fa98ec00640 1 -- 172.21.15.88:0/1528178838 wait complete. 2024-04-26T20:02:26.287 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:26.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.479+0000 7f7b6f200640 1 Processor -- start 2024-04-26T20:02:26.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.479+0000 7f7b6f200640 1 -- start start 2024-04-26T20:02:26.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.480+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b700b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6f200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7b700d0e70 0x7f7b700d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6f200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b70146a60 con 0x7f7b70160930 2024-04-26T20:02:26.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6f200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f7b7006e970 con 0x7f7b700d0e70 2024-04-26T20:02:26.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7b700d17f0 con 0x7f7b70146680 2024-04-26T20:02:26.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6e800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f7b70160930 0x7f7b70160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43470/0 (socket says 172.21.15.88:43470) 2024-04-26T20:02:26.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6e800640 1 -- 172.21.15.88:0/2450712685 learned_addr learned my addr 172.21.15.88:0/2450712685 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:26.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b67e00640 1 --2- 172.21.15.88:0/2450712685 >> v2:172.21.15.132:3300/0 conn(0x7f7b700d0e70 0x7f7b700d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6de00640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b700b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.481+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566601403 0 0) 0x7f7b70146a60 con 0x7f7b70160930 2024-04-26T20:02:26.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b54003520 con 0x7f7b70160930 2024-04-26T20:02:26.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1637079538 0 0) 0x7f7b54003520 con 0x7f7b70160930 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 msgr2=0x7f7b700b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b700b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 >> v2:172.21.15.132:3300/0 conn(0x7f7b700d0e70 msgr2=0x7f7b700d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 --2- 172.21.15.88:0/2450712685 >> v2:172.21.15.132:3300/0 conn(0x7f7b700d0e70 0x7f7b700d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7006dd60 con 0x7f7b70160930 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f7b680019f0 con 0x7f7b70160930 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 >> v1:172.21.15.88:6789/0 conn(0x7f7b70160930 legacy=0x7f7b70160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b540039b0 0x7f7b54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b54004d40 0x7f7b540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7b700d17f0 con 0x7f7b54004d40 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5400fdd0 con 0x7f7b540039b0 2024-04-26T20:02:26.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5400ff50 con 0x7f7b70146680 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.482+0000 7f7b6e800640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b54004d40 0x7f7b540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6de00640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b540039b0 0x7f7b54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b67e00640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6e800640 1 -- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 msgr2=0x7f7b54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6e800640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6e800640 1 -- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b540039b0 msgr2=0x7f7b54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6e800640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b540039b0 0x7f7b54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b6e800640 1 -- 172.21.15.88:0/2450712685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b54003520 con 0x7f7b54004d40 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.483+0000 7f7b67e00640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b70146680 0x7f7b54004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:26.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.484+0000 7f7b6e800640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b54004d40 0x7f7b540051c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800a950 tx=0x7f7b54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc302e330db5985 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.484+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800a4c0 con 0x7f7b54004d40 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.484+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6802f070 con 0x7f7b54004d40 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.484+0000 7f7b6d400640 1 -- 172.21.15.88:0/2450712685 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6802bd00 con 0x7f7b54004d40 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 -- 172.21.15.88:0/2450712685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b54004d40 msgr2=0x7f7b540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 --2- 172.21.15.88:0/2450712685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b54004d40 0x7f7b540051c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800a950 tx=0x7f7b54005830 comp rx=0 tx=0).stop 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6e800640 1 -- 172.21.15.88:0/2450712685 reap_dead start 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 -- 172.21.15.88:0/2450712685 shutdown_connections 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 -- 172.21.15.88:0/2450712685 >> 172.21.15.88:0/2450712685 conn(0x7f7b7006c740 msgr2=0x7f7b7006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 -- 172.21.15.88:0/2450712685 shutdown_connections 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.485+0000 7f7b6f200640 1 -- 172.21.15.88:0/2450712685 wait complete. 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.487+0000 7f7b6f200640 1 Processor -- start 2024-04-26T20:02:26.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.487+0000 7f7b6f200640 1 -- start start 2024-04-26T20:02:26.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b54004d40 0x7f7b700cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b70141fa0 0x7f7b7015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7006e600 con 0x7f7b700731f0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7006e320 con 0x7f7b70141fa0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6f200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7b700cd6a0 con 0x7f7b54004d40 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52658/0 (socket says 172.21.15.88:52658) 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b6e800640 1 -- 172.21.15.88:0/1385602484 learned_addr learned my addr 172.21.15.88:0/1385602484 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.488+0000 7f7b67e00640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b70141fa0 0x7f7b7015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 -- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b54004d40 msgr2=0x7f7b700cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b54004d40 0x7f7b700cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 -- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b70141fa0 msgr2=0x7f7b7015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b70141fa0 0x7f7b7015dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 -- 172.21.15.88:0/1385602484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7014a0a0 con 0x7f7b700731f0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6e800640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802cae0 tx=0x7f7b7006e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800a4c0 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b680019e0 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68002000 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.489+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7014a380 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.490+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7014a940 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.491+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b700693c0 con 0x7f7b700731f0 2024-04-26T20:02:26.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.491+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7b68046060 con 0x7f7b700731f0 2024-04-26T20:02:26.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.492+0000 7f7b66000640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 0x7f7b4c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.493+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f7b680abd00 con 0x7f7b700731f0 2024-04-26T20:02:26.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.493+0000 7f7b6de00640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 0x7f7b4c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.493+0000 7f7b6de00640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 0x7f7b4c06ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b60006fd0 tx=0x7f7b6000b2e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.497+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b680333d0 con 0x7f7b700731f0 2024-04-26T20:02:26.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.699+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b700b6040 con 0x7f7b700731f0 2024-04-26T20:02:26.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.702+0000 7f7b66000640 1 -- 172.21.15.88:0/1385602484 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7f7b68079fb0 con 0x7f7b700731f0 2024-04-26T20:02:26.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:26.705 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:26.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 msgr2=0x7f7b4c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 0x7f7b4c06ee40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b60006fd0 tx=0x7f7b6000b2e0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 msgr2=0x7f7b70073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802cae0 tx=0x7f7b7006e8e0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 shutdown_connections 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b700731f0 0x7f7b70073670 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b70141fa0 0x7f7b7015dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7b4c06c980 0x7f7b4c06ee40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 --2- 172.21.15.88:0/1385602484 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7b54004d40 0x7f7b700cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 >> 172.21.15.88:0/1385602484 conn(0x7f7b7006c740 msgr2=0x7f7b700ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:26.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 shutdown_connections 2024-04-26T20:02:26.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.705+0000 7f7b6f200640 1 -- 172.21.15.88:0/1385602484 wait complete. 2024-04-26T20:02:26.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:01:59.312423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:00.563428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:02:26.727 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:02:26.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.929+0000 7f67bfa00640 1 Processor -- start 2024-04-26T20:02:26.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.929+0000 7f67bfa00640 1 -- start start 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d19c0 0x7f67b80d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f67b80b6170 0x7f67b80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67b80d1da0 0x7f67b8162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f67b80d40d0 con 0x7f67b80d1da0 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f67b80d3830 con 0x7f67b80b6170 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bfa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f67b80d3b10 con 0x7f67b80d19c0 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bda00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67b80d1da0 0x7f67b8162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bda00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f67b80d1da0 0x7f67b8162640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52674/0 (socket says 172.21.15.88:52674) 2024-04-26T20:02:26.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.931+0000 7f67bda00640 1 -- 172.21.15.88:0/498016996 learned_addr learned my addr 172.21.15.88:0/498016996 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.932+0000 7f67bda00640 1 -- 172.21.15.88:0/498016996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d19c0 msgr2=0x7f67b80d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.932+0000 7f67bda00640 1 --2- 172.21.15.88:0/498016996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d19c0 0x7f67b80d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.932+0000 7f67bda00640 1 -- 172.21.15.88:0/498016996 >> v2:172.21.15.132:3300/0 conn(0x7f67b80b6170 msgr2=0x7f67b80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.932+0000 7f67bda00640 1 --2- 172.21.15.88:0/498016996 >> v2:172.21.15.132:3300/0 conn(0x7f67b80b6170 0x7f67b80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.932+0000 7f67bda00640 1 -- 172.21.15.88:0/498016996 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b80d3df0 con 0x7f67b80d1da0 2024-04-26T20:02:26.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.933+0000 7f67bda00640 1 --2- 172.21.15.88:0/498016996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80d1da0 0x7f67b8162640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67b809eb90 tx=0x7f67b002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9c149779bd733 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.933+0000 7f67b7400640 1 -- 172.21.15.88:0/498016996 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0036070 con 0x7f67b80d1da0 2024-04-26T20:02:26.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.933+0000 7f67b7400640 1 -- 172.21.15.88:0/498016996 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b0031600 con 0x7f67b80d1da0 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.933+0000 7f67b7400640 1 -- 172.21.15.88:0/498016996 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b00319d0 con 0x7f67b80d1da0 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 -- 172.21.15.88:0/498016996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80d1da0 msgr2=0x7f67b8162640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 --2- 172.21.15.88:0/498016996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80d1da0 0x7f67b8162640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f67b809eb90 tx=0x7f67b002a610 comp rx=0 tx=0).stop 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 -- 172.21.15.88:0/498016996 shutdown_connections 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 --2- 172.21.15.88:0/498016996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80d1da0 0x7f67b8162640 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 --2- 172.21.15.88:0/498016996 >> v2:172.21.15.132:3300/0 conn(0x7f67b80b6170 0x7f67b80c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 --2- 172.21.15.88:0/498016996 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d19c0 0x7f67b80d2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 -- 172.21.15.88:0/498016996 >> 172.21.15.88:0/498016996 conn(0x7f67b809e460 msgr2=0x7f67b80d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 -- 172.21.15.88:0/498016996 shutdown_connections 2024-04-26T20:02:26.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.934+0000 7f67bfa00640 1 -- 172.21.15.88:0/498016996 wait complete. 2024-04-26T20:02:26.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.935+0000 7f67bfa00640 1 Processor -- start 2024-04-26T20:02:26.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 -- start start 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b80d19c0 0x7f67b8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 0x7f67b80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67b80cfc30 con 0x7f67b80b6170 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f67b80c91b0 con 0x7f67b80d19c0 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.936+0000 7f67bfa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f67b8162b80 con 0x7f67b80d1da0 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52682/0 (socket says 172.21.15.88:52682) 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 -- 172.21.15.88:0/166844140 learned_addr learned my addr 172.21.15.88:0/166844140 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bda00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 0x7f67b80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67b7e00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b80d19c0 0x7f67b8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 -- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 msgr2=0x7f67b80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 0x7f67b80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 -- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b80d19c0 msgr2=0x7f67b8157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b80d19c0 0x7f67b8157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 -- 172.21.15.88:0/166844140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81581f0 con 0x7f67b80b6170 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bda00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 0x7f67b80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.937+0000 7f67bd000640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b4a0 tx=0x7f67ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.938+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac014070 con 0x7f67b80b6170 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.938+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ac010610 con 0x7f67b80b6170 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.938+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010a00 con 0x7f67b80b6170 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.938+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b8158490 con 0x7f67b80b6170 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.938+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b8158a50 con 0x7f67b80b6170 2024-04-26T20:02:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.939+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b8069650 con 0x7f67b80b6170 2024-04-26T20:02:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.940+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f67ac017070 con 0x7f67b80b6170 2024-04-26T20:02:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.941+0000 7f67b4c00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 0x7f679806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.941+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f67ac08b6d0 con 0x7f67b80b6170 2024-04-26T20:02:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.941+0000 7f67b7e00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 0x7f679806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.942+0000 7f67b7e00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 0x7f679806ee70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67b8160040 tx=0x7f67a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:26.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:26.946+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67ac059a10 con 0x7f67b80b6170 2024-04-26T20:02:27.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.147+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67b8162f80 con 0x7f67b80b6170 2024-04-26T20:02:27.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.148+0000 7f67b4c00640 1 -- 172.21.15.88:0/166844140 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f67ac059310 con 0x7f67b80b6170 2024-04-26T20:02:27.150 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:02:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 msgr2=0x7f679806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 0x7f679806ee70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67b8160040 tx=0x7f67a4008040 comp rx=0 tx=0).stop 2024-04-26T20:02:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 msgr2=0x7f67b81f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00b4a0 tx=0x7f67ac00b960 comp rx=0 tx=0).stop 2024-04-26T20:02:27.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 shutdown_connections 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f679806c9b0 0x7f679806ee70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f67b80d1da0 0x7f67b80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67b80d19c0 0x7f67b8157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 --2- 172.21.15.88:0/166844140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b80b6170 0x7f67b81f5b30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 >> 172.21.15.88:0/166844140 conn(0x7f67b809e460 msgr2=0x7f67b80c8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 shutdown_connections 2024-04-26T20:02:27.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.151+0000 7f67bfa00640 1 -- 172.21.15.88:0/166844140 wait complete. 2024-04-26T20:02:27.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:02:27.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.372+0000 7fb40dc00640 1 Processor -- start 2024-04-26T20:02:27.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.372+0000 7fb40dc00640 1 -- start start 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.373+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb4080b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.373+0000 7fb40dc00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb40806dfe0 0x7fb40815c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.373+0000 7fb40dc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40815d970 con 0x7fb40815cd70 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.373+0000 7fb40dc00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb4080b4fe0 con 0x7fb40806dfe0 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.373+0000 7fb40dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb4080b52c0 con 0x7fb40815c450 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb40806dfe0 0x7fb40815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb40806dfe0 0x7fb40815c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52694/0 (socket says 172.21.15.88:52694) 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 -- 172.21.15.88:0/3693525562 learned_addr learned my addr 172.21.15.88:0/3693525562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:27.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407e00640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb4080b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 -- 172.21.15.88:0/3693525562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 msgr2=0x7fb4080b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:27.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb4080b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 -- 172.21.15.88:0/3693525562 >> v1:172.21.15.132:6789/0 conn(0x7fb40815cd70 legacy=0x7fb40815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:27.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407400640 1 -- 172.21.15.88:0/3693525562 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4080b55a0 con 0x7fb40806dfe0 2024-04-26T20:02:27.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.374+0000 7fb407e00640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb4080b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:27.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.375+0000 7fb407400640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40815c830 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb408071b30 tx=0x7fb3f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163a32c934d02333 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb406a00640 1 -- 172.21.15.88:0/3693525562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4037070 con 0x7fb40806dfe0 2024-04-26T20:02:27.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb406a00640 1 -- 172.21.15.88:0/3693525562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f402ae50 con 0x7fb40806dfe0 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb406a00640 1 -- 172.21.15.88:0/3693525562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4032660 con 0x7fb40806dfe0 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 -- 172.21.15.88:0/3693525562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 msgr2=0x7fb40815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40815c830 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb408071b30 tx=0x7fb3f402a140 comp rx=0 tx=0).stop 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 -- 172.21.15.88:0/3693525562 shutdown_connections 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40815c830 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 --2- 172.21.15.88:0/3693525562 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb4080b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 -- 172.21.15.88:0/3693525562 >> 172.21.15.88:0/3693525562 conn(0x7fb4080b2900 msgr2=0x7fb4080b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 -- 172.21.15.88:0/3693525562 shutdown_connections 2024-04-26T20:02:27.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.376+0000 7fb40dc00640 1 -- 172.21.15.88:0/3693525562 wait complete. 2024-04-26T20:02:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.378+0000 7fb40dc00640 1 Processor -- start 2024-04-26T20:02:27.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.378+0000 7fb40dc00640 1 -- start start 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb40813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 0x7fb4080cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4080b5880 con 0x7fb40806dfe0 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb40815d1c0 con 0x7fb40815cd70 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb40dc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb4080b5a00 con 0x7fb40815c450 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.379+0000 7fb407e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52710/0 (socket says 172.21.15.88:52710) 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 -- 172.21.15.88:0/2241511674 learned_addr learned my addr 172.21.15.88:0/2241511674 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 -- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 msgr2=0x7fb40813e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb40813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 -- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 msgr2=0x7fb4080cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb40d200640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 0x7fb4080cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 0x7fb4080cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 -- 172.21.15.88:0/2241511674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4080d02f0 con 0x7fb40806dfe0 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb40d200640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 0x7fb4080cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:27.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.380+0000 7fb407e00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c7f0 tx=0x7fb3f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.381+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8015070 con 0x7fb40806dfe0 2024-04-26T20:02:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.381+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f8007a30 con 0x7fb40806dfe0 2024-04-26T20:02:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.381+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007e00 con 0x7fb40806dfe0 2024-04-26T20:02:27.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.381+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb408158350 con 0x7fb40806dfe0 2024-04-26T20:02:27.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.381+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb408158910 con 0x7fb40806dfe0 2024-04-26T20:02:27.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.382+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4080693c0 con 0x7fb40806dfe0 2024-04-26T20:02:27.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.383+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb3f8018070 con 0x7fb40806dfe0 2024-04-26T20:02:27.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.384+0000 7fb3ffe00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 0x7fb3e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.384+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fb3f808bd10 con 0x7fb40806dfe0 2024-04-26T20:02:27.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.384+0000 7fb407400640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 0x7fb3e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.385+0000 7fb407400640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 0x7fb3e806ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.388+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3f8059fd0 con 0x7fb40806dfe0 2024-04-26T20:02:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.589+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4080d16a0 con 0x7fb40806dfe0 2024-04-26T20:02:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.590+0000 7fb3ffe00640 1 -- 172.21.15.88:0/2241511674 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7fb3f80598d0 con 0x7fb40806dfe0 2024-04-26T20:02:27.592 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 msgr2=0x7fb3e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 0x7fb3e806ee40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402c000 comp rx=0 tx=0).stop 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 msgr2=0x7fb40813af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c7f0 tx=0x7fb3f800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 shutdown_connections 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb40815cd70 0x7fb4080cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb3e806c980 0x7fb3e806ee40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb40815c450 0x7fb40813e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 --2- 172.21.15.88:0/2241511674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40806dfe0 0x7fb40813af60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 >> 172.21.15.88:0/2241511674 conn(0x7fb4080b2900 msgr2=0x7fb40806fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:27.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 shutdown_connections 2024-04-26T20:02:27.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.593+0000 7fb40dc00640 1 -- 172.21.15.88:0/2241511674 wait complete. 2024-04-26T20:02:27.614 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:02:27.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.814+0000 7fd55fe00640 1 Processor -- start 2024-04-26T20:02:27.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.815+0000 7fd55fe00640 1 -- start start 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd55806ce80 0x7fd5580b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5580b5ca0 0x7fd5580b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55fe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5580af800 con 0x7fd55815d100 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55fe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd5580ae340 con 0x7fd55806ce80 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd5580afaa0 con 0x7fd5580b5ca0 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55ca00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd55806ce80 0x7fd5580b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55ca00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd55806ce80 0x7fd5580b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52718/0 (socket says 172.21.15.88:52718) 2024-04-26T20:02:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.816+0000 7fd55ca00640 1 -- 172.21.15.88:0/3141905183 learned_addr learned my addr 172.21.15.88:0/3141905183 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:27.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.817+0000 7fd55de00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5580b5ca0 0x7fd5580b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.817+0000 7fd55ca00640 1 -- 172.21.15.88:0/3141905183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5580b5ca0 msgr2=0x7fd5580b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.817+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5580b5ca0 0x7fd5580b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.817+0000 7fd55ca00640 1 -- 172.21.15.88:0/3141905183 >> v1:172.21.15.132:6789/0 conn(0x7fd55815d100 legacy=0x7fd55806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.817+0000 7fd55ca00640 1 -- 172.21.15.88:0/3141905183 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5580afd80 con 0x7fd55806ce80 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.818+0000 7fd55ca00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55806ce80 0x7fd5580b3c30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd54402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630910e1c443f383 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.818+0000 7fd557e00640 1 -- 172.21.15.88:0/3141905183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544037070 con 0x7fd55806ce80 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.818+0000 7fd557e00640 1 -- 172.21.15.88:0/3141905183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd544032440 con 0x7fd55806ce80 2024-04-26T20:02:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.818+0000 7fd557e00640 1 -- 172.21.15.88:0/3141905183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544032810 con 0x7fd55806ce80 2024-04-26T20:02:27.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 -- 172.21.15.88:0/3141905183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55806ce80 msgr2=0x7fd5580b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55806ce80 0x7fd5580b3c30 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd544009870 tx=0x7fd54402a210 comp rx=0 tx=0).stop 2024-04-26T20:02:27.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 -- 172.21.15.88:0/3141905183 shutdown_connections 2024-04-26T20:02:27.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd5580b5ca0 0x7fd5580b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 --2- 172.21.15.88:0/3141905183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd55806ce80 0x7fd5580b3c30 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 -- 172.21.15.88:0/3141905183 >> 172.21.15.88:0/3141905183 conn(0x7fd5580b2d50 msgr2=0x7fd5580b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 -- 172.21.15.88:0/3141905183 shutdown_connections 2024-04-26T20:02:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.819+0000 7fd55fe00640 1 -- 172.21.15.88:0/3141905183 wait complete. 2024-04-26T20:02:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.821+0000 7fd55fe00640 1 Processor -- start 2024-04-26T20:02:27.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.821+0000 7fd55fe00640 1 -- start start 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55806ce80 0x7fd5581442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 0x7fd5581576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd55809ee50 con 0x7fd5580b5ca0 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd5580b0340 con 0x7fd55815d100 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd558160b70 con 0x7fd55806ce80 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52724/0 (socket says 172.21.15.88:52724) 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.822+0000 7fd55ca00640 1 -- 172.21.15.88:0/1756045871 learned_addr learned my addr 172.21.15.88:0/1756045871 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55de00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 0x7fd5581576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55d400640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55806ce80 0x7fd5581442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 -- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55806ce80 msgr2=0x7fd5581442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55806ce80 0x7fd5581442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 -- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 msgr2=0x7fd5581576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 0x7fd5581576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 -- 172.21.15.88:0/1756045871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd558144800 con 0x7fd5580b5ca0 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55de00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 0x7fd5581576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:27.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.823+0000 7fd55ca00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd5440097c0 tx=0x7fd5440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.824+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544041020 con 0x7fd5580b5ca0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.824+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd544030970 con 0x7fd5580b5ca0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.824+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55809d740 con 0x7fd5580b5ca0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.824+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544037040 con 0x7fd5580b5ca0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.824+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55809dd00 con 0x7fd5580b5ca0 2024-04-26T20:02:27.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.825+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd558069650 con 0x7fd5580b5ca0 2024-04-26T20:02:27.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.826+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd544044020 con 0x7fd5580b5ca0 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.827+0000 7fd555600640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 0x7fd53806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.827+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fd5440ac560 con 0x7fd5580b5ca0 2024-04-26T20:02:27.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.827+0000 7fd55d400640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 0x7fd53806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:27.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.828+0000 7fd55d400640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 0x7fd53806ee40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd54c007bf0 tx=0x7fd54c007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:27.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:27.831+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd54404b030 con 0x7fd5580b5ca0 2024-04-26T20:02:28.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.031+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd558073d00 con 0x7fd5580b5ca0 2024-04-26T20:02:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.032+0000 7fd555600640 1 -- 172.21.15.88:0/1756045871 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7fd54407a930 con 0x7fd5580b5ca0 2024-04-26T20:02:28.034 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:02:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.034+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 msgr2=0x7fd53806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 0x7fd53806ee40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd54c007bf0 tx=0x7fd54c007240 comp rx=0 tx=0).stop 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 msgr2=0x7fd558157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd5440097c0 tx=0x7fd5440046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 shutdown_connections 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd55815d100 0x7fd5581576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5580b5ca0 0x7fd558157170 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd53806c980 0x7fd53806ee40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 --2- 172.21.15.88:0/1756045871 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd55806ce80 0x7fd5581442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 >> 172.21.15.88:0/1756045871 conn(0x7fd5580b2d50 msgr2=0x7fd5580b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 shutdown_connections 2024-04-26T20:02:28.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.035+0000 7fd55fe00640 1 -- 172.21.15.88:0/1756045871 wait complete. 2024-04-26T20:02:28.056 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:28.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.246+0000 7f0171a00640 1 Processor -- start 2024-04-26T20:02:28.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.246+0000 7f0171a00640 1 -- start start 2024-04-26T20:02:28.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.248+0000 7f0171a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.248+0000 7f0171a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f016c0cd650 0x7f016c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.248+0000 7f0171a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c0af9a0 con 0x7f016c15cdf0 2024-04-26T20:02:28.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.248+0000 7f0171a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f016c0ae2c0 con 0x7f016c0cd650 2024-04-26T20:02:28.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.248+0000 7f0171a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f016c0b05b0 con 0x7f016c06ce80 2024-04-26T20:02:28.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:33700/0 (socket says 172.21.15.88:33700) 2024-04-26T20:02:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 -- 172.21.15.88:0/2684096350 learned_addr learned my addr 172.21.15.88:0/2684096350 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016b400640 1 --2- 172.21.15.88:0/2684096350 >> v2:172.21.15.132:3300/0 conn(0x7f016c0cd650 0x7f016c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 -- 172.21.15.88:0/2684096350 >> v2:172.21.15.132:3300/0 conn(0x7f016c0cd650 msgr2=0x7f016c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 --2- 172.21.15.88:0/2684096350 >> v2:172.21.15.132:3300/0 conn(0x7f016c0cd650 0x7f016c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 -- 172.21.15.88:0/2684096350 >> v1:172.21.15.132:6789/0 conn(0x7f016c15cdf0 legacy=0x7f016c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.249+0000 7f016a000640 1 -- 172.21.15.88:0/2684096350 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c0afc80 con 0x7f016c06ce80 2024-04-26T20:02:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.251+0000 7f016a000640 1 --2- 172.21.15.88:0/2684096350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0158009870 tx=0x7f015802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5d2c477e9b3b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.251+0000 7f0169600640 1 -- 172.21.15.88:0/2684096350 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158037070 con 0x7f016c06ce80 2024-04-26T20:02:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.251+0000 7f0169600640 1 -- 172.21.15.88:0/2684096350 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0158032440 con 0x7f016c06ce80 2024-04-26T20:02:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.251+0000 7f0169600640 1 -- 172.21.15.88:0/2684096350 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158032810 con 0x7f016c06ce80 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 -- 172.21.15.88:0/2684096350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 msgr2=0x7f016c0add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 --2- 172.21.15.88:0/2684096350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0158009870 tx=0x7f015802a250 comp rx=0 tx=0).stop 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 -- 172.21.15.88:0/2684096350 shutdown_connections 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 --2- 172.21.15.88:0/2684096350 >> v2:172.21.15.132:3300/0 conn(0x7f016c0cd650 0x7f016c0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 --2- 172.21.15.88:0/2684096350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c06ce80 0x7f016c0add80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 -- 172.21.15.88:0/2684096350 >> 172.21.15.88:0/2684096350 conn(0x7f016c0b2d50 msgr2=0x7f016c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 -- 172.21.15.88:0/2684096350 shutdown_connections 2024-04-26T20:02:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.252+0000 7f0171a00640 1 -- 172.21.15.88:0/2684096350 wait complete. 2024-04-26T20:02:28.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.254+0000 7f0171a00640 1 Processor -- start 2024-04-26T20:02:28.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.254+0000 7f0171a00640 1 -- start start 2024-04-26T20:02:28.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f016c06ce80 0x7f016c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c0cd650 0x7f016c1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f016c15d1d0 con 0x7f016c15cdf0 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0b0240 con 0x7f016c06ce80 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.255+0000 7f0171a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f016c15d350 con 0x7f016c0cd650 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52726/0 (socket says 172.21.15.88:52726) 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 -- 172.21.15.88:0/2742334058 learned_addr learned my addr 172.21.15.88:0/2742334058 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016a000640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c0cd650 0x7f016c1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016aa00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f016c06ce80 0x7f016c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 -- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c0cd650 msgr2=0x7f016c1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c0cd650 0x7f016c1488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 -- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f016c06ce80 msgr2=0x7f016c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f016c06ce80 0x7f016c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.256+0000 7f016b400640 1 -- 172.21.15.88:0/2742334058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c1493e0 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f016b400640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e30 tx=0x7f016400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164018020 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0164014480 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c1496c0 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164014870 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.257+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c149f60 con 0x7f016c15cdf0 2024-04-26T20:02:28.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.258+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c149890 con 0x7f016c15cdf0 2024-04-26T20:02:28.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.259+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0164005d10 con 0x7f016c15cdf0 2024-04-26T20:02:28.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.260+0000 7f0162a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 0x7f014c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.260+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f016408ccd0 con 0x7f016c15cdf0 2024-04-26T20:02:28.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.261+0000 7f016aa00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 0x7f014c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.261+0000 7f016aa00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 0x7f014c06ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f015c001640 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.265+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0164022030 con 0x7f016c15cdf0 2024-04-26T20:02:28.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.510+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016c069540 con 0x7f016c15cdf0 2024-04-26T20:02:28.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.511+0000 7f0162a00640 1 -- 172.21.15.88:0/2742334058 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5181 (secure 0 0 0) 0x7f0164010040 con 0x7f016c15cdf0 2024-04-26T20:02:28.513 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:28.513 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:28.513 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-04-26T20:02:28.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 msgr2=0x7f014c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 0x7f014c06ee40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f015c001640 tx=0x7f015c009040 comp rx=0 tx=0).stop 2024-04-26T20:02:28.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 msgr2=0x7f016c09dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0164009e30 tx=0x7f016400c490 comp rx=0 tx=0).stop 2024-04-26T20:02:28.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 shutdown_connections 2024-04-26T20:02:28.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016c15cdf0 0x7f016c09dc20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f014c06c980 0x7f014c06ee40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f016c0cd650 0x7f016c1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 --2- 172.21.15.88:0/2742334058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f016c06ce80 0x7f016c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 >> 172.21.15.88:0/2742334058 conn(0x7f016c0b2d50 msgr2=0x7f016c06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:28.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 shutdown_connections 2024-04-26T20:02:28.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.514+0000 7f0171a00640 1 -- 172.21.15.88:0/2742334058 wait complete. 2024-04-26T20:02:28.534 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:28.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.729+0000 7f4807200640 1 Processor -- start 2024-04-26T20:02:28.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.730+0000 7f4807200640 1 -- start start 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4807200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f480815f1b0 0x7f48080d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4807200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 0x7f48080b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4807200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48080d3f50 con 0x7f4808162b00 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4807200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f48080d36b0 con 0x7f480815f1b0 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4807200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f48080d3990 con 0x7f4808163650 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4805e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f480815f1b0 0x7f48080d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4805400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4808162b00 0x7f4808162f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:43484/0 (socket says 172.21.15.88:43484) 2024-04-26T20:02:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.731+0000 7f4805400640 1 -- 172.21.15.88:0/1120626894 learned_addr learned my addr 172.21.15.88:0/1120626894 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4806800640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 0x7f48080b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4805e00640 1 -- 172.21.15.88:0/1120626894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 msgr2=0x7f48080b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4804a00640 1 -- 172.21.15.88:0/1120626894 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291395807 0 0) 0x7f48080d3f50 con 0x7f4808162b00 2024-04-26T20:02:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4805e00640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 0x7f48080b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4805e00640 1 -- 172.21.15.88:0/1120626894 >> v1:172.21.15.88:6789/0 conn(0x7f4808162b00 legacy=0x7f4808162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4805e00640 1 -- 172.21.15.88:0/1120626894 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48080d3c70 con 0x7f480815f1b0 2024-04-26T20:02:28.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.732+0000 7f4806800640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 0x7f48080b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.733+0000 7f4805e00640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480815f1b0 0x7f48080d5990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009d90 tx=0x7f47f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c197877c8025368e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4804a00640 1 -- 172.21.15.88:0/1120626894 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8007740 con 0x7f480815f1b0 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4804a00640 1 -- 172.21.15.88:0/1120626894 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f80078e0 con 0x7f480815f1b0 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4804a00640 1 -- 172.21.15.88:0/1120626894 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8007cb0 con 0x7f480815f1b0 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 -- 172.21.15.88:0/1120626894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480815f1b0 msgr2=0x7f48080d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480815f1b0 0x7f48080d5990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009d90 tx=0x7f47f802a660 comp rx=0 tx=0).stop 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 -- 172.21.15.88:0/1120626894 shutdown_connections 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4808163650 0x7f48080b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 --2- 172.21.15.88:0/1120626894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480815f1b0 0x7f48080d5990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 -- 172.21.15.88:0/1120626894 >> 172.21.15.88:0/1120626894 conn(0x7f480809e230 msgr2=0x7f48080d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 -- 172.21.15.88:0/1120626894 shutdown_connections 2024-04-26T20:02:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.734+0000 7f4807200640 1 -- 172.21.15.88:0/1120626894 wait complete. 2024-04-26T20:02:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.736+0000 7f4807200640 1 Processor -- start 2024-04-26T20:02:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.736+0000 7f4807200640 1 -- start start 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 0x7f4808161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4808162b00 0x7f4808148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48080b87f0 con 0x7f4808163650 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f48080d33a0 con 0x7f4808162b00 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4807200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f48080b8970 con 0x7f480815f1b0 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.737+0000 7f4806800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52742/0 (socket says 172.21.15.88:52742) 2024-04-26T20:02:28.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 -- 172.21.15.88:0/1696681106 learned_addr learned my addr 172.21.15.88:0/1696681106 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4805e00640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 0x7f4808161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 -- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 msgr2=0x7f4808161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4805400640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4808162b00 0x7f4808148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 0x7f4808161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 -- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4808162b00 msgr2=0x7f4808148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4808162b00 0x7f4808148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 -- 172.21.15.88:0/1696681106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48080cabf0 con 0x7f4808163650 2024-04-26T20:02:28.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4805e00640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 0x7f4808161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.738+0000 7f4806800640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f480000c610 tx=0x7f480000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.739+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800015070 con 0x7f4808163650 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.739+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800004240 con 0x7f4808163650 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.739+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48080caed0 con 0x7f4808163650 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.739+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800003610 con 0x7f4808163650 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.739+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48081533c0 con 0x7f4808163650 2024-04-26T20:02:28.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.740+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080693c0 con 0x7f4808163650 2024-04-26T20:02:28.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.741+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f48000037b0 con 0x7f4808163650 2024-04-26T20:02:28.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.742+0000 7f47fe000640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 0x7f47e406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:28.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.743+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f480008bbf0 con 0x7f4808163650 2024-04-26T20:02:28.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.743+0000 7f4805e00640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 0x7f47e406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:28.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.743+0000 7f4805e00640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 0x7f47e406ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47f8034000 tx=0x7f47f802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:28.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.747+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4800059e30 con 0x7f4808163650 2024-04-26T20:02:28.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.948+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f480808d090 con 0x7f4808163650 2024-04-26T20:02:28.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.950+0000 7f47fe000640 1 -- 172.21.15.88:0/1696681106 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7f4800059730 con 0x7f4808163650 2024-04-26T20:02:28.952 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:28.952 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 msgr2=0x7f47e406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 0x7f47e406ee40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f47f8034000 tx=0x7f47f802f000 comp rx=0 tx=0).stop 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 msgr2=0x7f4808148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f480000c610 tx=0x7f480000cad0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 shutdown_connections 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4808163650 0x7f4808148ed0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4808162b00 0x7f4808148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47e406c980 0x7f47e406ee40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 --2- 172.21.15.88:0/1696681106 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480815f1b0 0x7f4808161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:28.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 >> 172.21.15.88:0/1696681106 conn(0x7f480809e230 msgr2=0x7f48080d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:28.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 shutdown_connections 2024-04-26T20:02:28.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:28.953+0000 7f4807200640 1 -- 172.21.15.88:0/1696681106 wait complete. 2024-04-26T20:02:28.974 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:02:28.974 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:29.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.172+0000 7f4812800640 1 Processor -- start 2024-04-26T20:02:29.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.173+0000 7f4812800640 1 -- start start 2024-04-26T20:02:29.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.174+0000 7f4812800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f480c0d0e70 0x7f480c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.174+0000 7f4812800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5660 0x7f480c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.174+0000 7f4812800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f480c0d1790 con 0x7f480c0b5d40 2024-04-26T20:02:29.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.174+0000 7f4812800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f480c0727f0 con 0x7f480c0d0e70 2024-04-26T20:02:29.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.174+0000 7f4812800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f480c0b4a10 con 0x7f480c0b5660 2024-04-26T20:02:29.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.175+0000 7f4811400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f480c0d0e70 0x7f480c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.175+0000 7f4811400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f480c0d0e70 0x7f480c0d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:58174/0 (socket says 172.21.15.88:58174) 2024-04-26T20:02:29.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.175+0000 7f4811400640 1 -- 172.21.15.88:0/772222800 learned_addr learned my addr 172.21.15.88:0/772222800 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:29.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.175+0000 7f4810a00640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5660 0x7f480c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.176+0000 7f4811400640 1 -- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5660 msgr2=0x7f480c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:29.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.176+0000 7f4811400640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5660 0x7f480c0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.176+0000 7f4811400640 1 -- 172.21.15.88:0/772222800 >> v1:172.21.15.132:6789/0 conn(0x7f480c0b5d40 legacy=0x7f480c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.176+0000 7f4811400640 1 -- 172.21.15.88:0/772222800 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c0b4cf0 con 0x7f480c0d0e70 2024-04-26T20:02:29.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.177+0000 7f4811400640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0d0e70 0x7f480c0d1250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4800009870 tx=0x7f480002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a826f5828b03efe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.178+0000 7f480be00640 1 -- 172.21.15.88:0/772222800 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800037070 con 0x7f480c0d0e70 2024-04-26T20:02:29.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.178+0000 7f480be00640 1 -- 172.21.15.88:0/772222800 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800032440 con 0x7f480c0d0e70 2024-04-26T20:02:29.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.178+0000 7f480be00640 1 -- 172.21.15.88:0/772222800 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800032810 con 0x7f480c0d0e70 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 -- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0d0e70 msgr2=0x7f480c0d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0d0e70 0x7f480c0d1250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4800009870 tx=0x7f480002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 -- 172.21.15.88:0/772222800 shutdown_connections 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5660 0x7f480c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 --2- 172.21.15.88:0/772222800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0d0e70 0x7f480c0d1250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 -- 172.21.15.88:0/772222800 >> 172.21.15.88:0/772222800 conn(0x7f480c06c740 msgr2=0x7f480c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 -- 172.21.15.88:0/772222800 shutdown_connections 2024-04-26T20:02:29.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.179+0000 7f4812800640 1 -- 172.21.15.88:0/772222800 wait complete. 2024-04-26T20:02:29.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.181+0000 7f4812800640 1 Processor -- start 2024-04-26T20:02:29.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.181+0000 7f4812800640 1 -- start start 2024-04-26T20:02:29.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0b5660 0x7f480c14e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5d40 0x7f480c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0b05b0 con 0x7f480c0d0e70 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f480c0b0430 con 0x7f480c0b5660 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4812800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f480c15f1a0 con 0x7f480c0b5d40 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4811e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4811e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57982/0 (socket says 172.21.15.88:57982) 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4811e00640 1 -- 172.21.15.88:0/2338910476 learned_addr learned my addr 172.21.15.88:0/2338910476 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4810a00640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5d40 0x7f480c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.183+0000 7f4811400640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0b5660 0x7f480c14e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 -- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5d40 msgr2=0x7f480c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5d40 0x7f480c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 -- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0b5660 msgr2=0x7f480c14e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0b5660 0x7f480c14e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 -- 172.21.15.88:0/2338910476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c15e9a0 con 0x7f480c0d0e70 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4811e00640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f480400c6b0 tx=0x7f480400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804003b70 con 0x7f480c0d0e70 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804003d30 con 0x7f480c0d0e70 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1f52f0 con 0x7f480c0d0e70 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.184+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804005310 con 0x7f480c0d0e70 2024-04-26T20:02:29.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.185+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1f58b0 con 0x7f480c0d0e70 2024-04-26T20:02:29.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.186+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c0693c0 con 0x7f480c0d0e70 2024-04-26T20:02:29.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.187+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f48040054b0 con 0x7f480c0d0e70 2024-04-26T20:02:29.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.188+0000 7f4809600640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 0x7f47ec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.188+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f480408b8a0 con 0x7f480c0d0e70 2024-04-26T20:02:29.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.188+0000 7f4811400640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 0x7f47ec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.189+0000 7f4811400640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 0x7f47ec06ee40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.192+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4804059690 con 0x7f480c0d0e70 2024-04-26T20:02:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.438+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f480c0afd40 con 0x7f480c0d0e70 2024-04-26T20:02:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.439+0000 7f4809600640 1 -- 172.21.15.88:0/2338910476 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5181 (secure 0 0 0) 0x7f480405d990 con 0x7f480c0d0e70 2024-04-26T20:02:29.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:29.441 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 msgr2=0x7f47ec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 0x7f47ec06ee40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002e000 comp rx=0 tx=0).stop 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 msgr2=0x7f480c0b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f480400c6b0 tx=0x7f480400cb70 comp rx=0 tx=0).stop 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 shutdown_connections 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f480c0d0e70 0x7f480c0b1b00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f47ec06c980 0x7f47ec06ee40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f480c0b5d40 0x7f480c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 --2- 172.21.15.88:0/2338910476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f480c0b5660 0x7f480c14e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 >> 172.21.15.88:0/2338910476 conn(0x7f480c06c740 msgr2=0x7f480c152810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:29.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 shutdown_connections 2024-04-26T20:02:29.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.442+0000 7f4812800640 1 -- 172.21.15.88:0/2338910476 wait complete. 2024-04-26T20:02:29.463 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:29.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.652+0000 7f7446400640 1 Processor -- start 2024-04-26T20:02:29.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.652+0000 7f7446400640 1 -- start start 2024-04-26T20:02:29.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f7446400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f744006c770 0x7f744006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f7446400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7440073f40 con 0x7f744015cb70 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f7446400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f744006d090 con 0x7f744015cfd0 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f7446400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f74400af460 con 0x7f744006c770 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f743f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f744006c770 0x7f744006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f743f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f744006c770 0x7f744006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57996/0 (socket says 172.21.15.88:57996) 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.654+0000 7f743f400640 1 -- 172.21.15.88:0/3581245021 learned_addr learned my addr 172.21.15.88:0/3581245021 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.653+0000 7f7437e00640 1 --1- 172.21.15.88:0/3581245021 >> v1:172.21.15.88:6789/0 conn(0x7f744015cb70 0x7f7440073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35100/0 (socket says 172.21.15.88:35100) 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.654+0000 7f743f400640 1 -- 172.21.15.88:0/3581245021 >> v1:172.21.15.132:6789/0 conn(0x7f744015cfd0 legacy=0x7f7440073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.654+0000 7f743f400640 1 -- 172.21.15.88:0/3581245021 >> v1:172.21.15.88:6789/0 conn(0x7f744015cb70 legacy=0x7f7440073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.654+0000 7f743f400640 1 -- 172.21.15.88:0/3581245021 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74400ae850 con 0x7f744006c770 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.655+0000 7f743f400640 1 --2- 172.21.15.88:0/3581245021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744006c770 0x7f744006cb50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7430009870 tx=0x7f743002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c20ef859c18443f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.655+0000 7f743ea00640 1 -- 172.21.15.88:0/3581245021 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430037070 con 0x7f744006c770 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.655+0000 7f743ea00640 1 -- 172.21.15.88:0/3581245021 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7430032440 con 0x7f744006c770 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.655+0000 7f743ea00640 1 -- 172.21.15.88:0/3581245021 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430032810 con 0x7f744006c770 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 -- 172.21.15.88:0/3581245021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744006c770 msgr2=0x7f744006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 --2- 172.21.15.88:0/3581245021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744006c770 0x7f744006cb50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7430009870 tx=0x7f743002a2a0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 -- 172.21.15.88:0/3581245021 shutdown_connections 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 --2- 172.21.15.88:0/3581245021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744006c770 0x7f744006cb50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 -- 172.21.15.88:0/3581245021 >> 172.21.15.88:0/3581245021 conn(0x7f74400b2d50 msgr2=0x7f74400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:29.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 -- 172.21.15.88:0/3581245021 shutdown_connections 2024-04-26T20:02:29.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.656+0000 7f7446400640 1 -- 172.21.15.88:0/3581245021 wait complete. 2024-04-26T20:02:29.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.658+0000 7f7446400640 1 Processor -- start 2024-04-26T20:02:29.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.658+0000 7f7446400640 1 -- start start 2024-04-26T20:02:29.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f744006c770 0x7f7440150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f744015cfd0 0x7f7440072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74400af0f0 con 0x7f744015cb70 2024-04-26T20:02:29.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f74400aee10 con 0x7f744015cfd0 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7446400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f744015c600 con 0x7f744006c770 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7437e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7437e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58006/0 (socket says 172.21.15.88:58006) 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f7437e00640 1 -- 172.21.15.88:0/2990582807 learned_addr learned my addr 172.21.15.88:0/2990582807 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f743f400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f744006c770 0x7f7440150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.659+0000 7f743fe00640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f744015cfd0 0x7f7440072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 -- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f744006c770 msgr2=0x7f7440150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f744006c770 0x7f7440150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 -- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f744015cfd0 msgr2=0x7f7440072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f744015cfd0 0x7f7440072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 -- 172.21.15.88:0/2990582807 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74401513b0 con 0x7f744015cb70 2024-04-26T20:02:29.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437e00640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f74280069a0 tx=0x7f742800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428017070 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74280064e0 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74280049e0 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74401f8880 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.660+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74401f8e40 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.661+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440069540 con 0x7f744015cb70 2024-04-26T20:02:29.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.662+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f742801a070 con 0x7f744015cb70 2024-04-26T20:02:29.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.664+0000 7f7437400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 0x7f741c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:29.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.664+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f742808b060 con 0x7f744015cb70 2024-04-26T20:02:29.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.664+0000 7f743f400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 0x7f741c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:29.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.665+0000 7f743f400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 0x7f741c06ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7430034000 tx=0x7f743002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:29.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.668+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74280593a0 con 0x7f744015cb70 2024-04-26T20:02:29.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.910+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74400b3340 con 0x7f744015cb70 2024-04-26T20:02:29.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.911+0000 7f7437400640 1 -- 172.21.15.88:0/2990582807 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5181 (secure 0 0 0) 0x7f7428058ca0 con 0x7f744015cb70 2024-04-26T20:02:29.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:29.914 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:01.583597+0000","modified":"2024-04-26T20:02:04.646446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5712":{"gid":5712,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6837/649375356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":649375356},{"type":"v1","addr":"172.21.15.88:6837","nonce":649375356}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5712,"qdb_cluster":[5712]},"id":4}]} 2024-04-26T20:02:29.914 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-04-26T20:02:29.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 msgr2=0x7f741c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 0x7f741c06ee40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7430034000 tx=0x7f743002e000 comp rx=0 tx=0).stop 2024-04-26T20:02:29.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 msgr2=0x7f74400725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:29.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f74280069a0 tx=0x7f742800c960 comp rx=0 tx=0).stop 2024-04-26T20:02:29.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 shutdown_connections 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f744015cfd0 0x7f7440072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744015cb70 0x7f74400725b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f741c06c980 0x7f741c06ee40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 --2- 172.21.15.88:0/2990582807 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f744006c770 0x7f7440150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.914+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 >> 172.21.15.88:0/2990582807 conn(0x7f74400b2d50 msgr2=0x7f74400b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.915+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 shutdown_connections 2024-04-26T20:02:29.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:29.915+0000 7f7446400640 1 -- 172.21.15.88:0/2990582807 wait complete. 2024-04-26T20:02:29.935 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:30.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.138+0000 7f28ada00640 1 Processor -- start 2024-04-26T20:02:30.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.138+0000 7f28ada00640 1 -- start start 2024-04-26T20:02:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.138+0000 7f28ada00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 0x7f28a806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28ada00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a8073f40 con 0x7f28a815cb70 2024-04-26T20:02:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28ada00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28a806d090 con 0x7f28a815cfd0 2024-04-26T20:02:30.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28ada00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f28a80af460 con 0x7f28a806c770 2024-04-26T20:02:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28a6000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f28a815cb70 0x7f28a8073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35104/0 (socket says 172.21.15.88:35104) 2024-04-26T20:02:30.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28a6000640 1 -- 172.21.15.88:0/182828199 learned_addr learned my addr 172.21.15.88:0/182828199 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28a6a00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 0x7f28a806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425395179 0 0) 0x7f28a8073f40 con 0x7f28a815cb70 2024-04-26T20:02:30.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.139+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f288c003420 con 0x7f28a815cb70 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1524834262 0 0) 0x7f288c003420 con 0x7f28a815cb70 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 msgr2=0x7f28a806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 0x7f28a806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 >> v1:172.21.15.132:6789/0 conn(0x7f28a815cfd0 legacy=0x7f28a8073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80ae850 con 0x7f28a815cb70 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f2894001670 con 0x7f28a815cb70 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 >> v1:172.21.15.88:6789/0 conn(0x7f28a815cb70 legacy=0x7f28a8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c004ac0 0x7f288c004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c0036a0 0x7f288c003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 0x7f288c00fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80af460 con 0x7f288c004ac0 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f288c0044a0 con 0x7f288c0036a0 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f288c0100b0 con 0x7f28a806c770 2024-04-26T20:02:30.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.140+0000 7f28a6a00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c004ac0 0x7f288c004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.141+0000 7f28a6a00640 1 -- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 msgr2=0x7f288c00fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.141+0000 7f28a6a00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a806c770 0x7f288c00fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.141+0000 7f28a6a00640 1 -- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c0036a0 msgr2=0x7f288c003b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.141+0000 7f28a6a00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c0036a0 0x7f288c003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.141+0000 7f28a6a00640 1 -- 172.21.15.88:0/182828199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c003420 con 0x7f288c004ac0 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.142+0000 7f28a6a00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c004ac0 0x7f288c004ee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2898009af0 tx=0x7f288c004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbc03865a27888a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.142+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898036070 con 0x7f288c004ac0 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.142+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2898031400 con 0x7f288c004ac0 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.142+0000 7f28a5600640 1 -- 172.21.15.88:0/182828199 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898009690 con 0x7f288c004ac0 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 -- 172.21.15.88:0/182828199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c004ac0 msgr2=0x7f288c004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 --2- 172.21.15.88:0/182828199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f288c004ac0 0x7f288c004ee0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2898009af0 tx=0x7f288c004720 comp rx=0 tx=0).stop 2024-04-26T20:02:30.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28a7400640 1 -- 172.21.15.88:0/182828199 reap_dead start 2024-04-26T20:02:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 -- 172.21.15.88:0/182828199 shutdown_connections 2024-04-26T20:02:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 -- 172.21.15.88:0/182828199 >> 172.21.15.88:0/182828199 conn(0x7f28a80b2d50 msgr2=0x7f28a80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 -- 172.21.15.88:0/182828199 shutdown_connections 2024-04-26T20:02:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.143+0000 7f28ada00640 1 -- 172.21.15.88:0/182828199 wait complete. 2024-04-26T20:02:30.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.145+0000 7f28ada00640 1 Processor -- start 2024-04-26T20:02:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.145+0000 7f28ada00640 1 -- start start 2024-04-26T20:02:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 0x7f28a80cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a809d030 0x7f28a80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80af0f0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f28a80aee10 con 0x7f288c004ac0 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28ada00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f28a815c600 con 0x7f28a809d030 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58018/0 (socket says 172.21.15.88:58018) 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 -- 172.21.15.88:0/979525986 learned_addr learned my addr 172.21.15.88:0/979525986 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a6a00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 0x7f28a80cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 -- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a809d030 msgr2=0x7f28a80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a809d030 0x7f28a80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 -- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 msgr2=0x7f28a80cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 0x7f28a80cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a7400640 1 -- 172.21.15.88:0/979525986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a80731e0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.146+0000 7f28a6a00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 0x7f28a80cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f28a7400640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f288c004a90 tx=0x7f28a000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0017070 con 0x7f28a80cd6e0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a0005d40 con 0x7f28a80cd6e0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a80734a0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000a850 con 0x7f28a80cd6e0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.147+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a80d0310 con 0x7f28a80cd6e0 2024-04-26T20:02:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.148+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a815bdc0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.149+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f28a001a070 con 0x7f28a80cd6e0 2024-04-26T20:02:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.151+0000 7f289ea00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 0x7f288806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.151+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f28a008aef0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.151+0000 7f28a6a00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 0x7f288806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.151+0000 7f28a6a00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 0x7f288806ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2898033000 tx=0x7f2898008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.155+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28a815bdc0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.355+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28a8069540 con 0x7f28a80cd6e0 2024-04-26T20:02:30.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.357+0000 7f289ea00640 1 -- 172.21.15.88:0/979525986 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16653 (secure 0 0 0) 0x7f28a00592c0 con 0x7f28a80cd6e0 2024-04-26T20:02:30.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:30.599 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:03.132365+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:01:59.312423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:00.563428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:30.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 msgr2=0x7f288806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 0x7f288806ee40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2898033000 tx=0x7f2898008030 comp rx=0 tx=0).stop 2024-04-26T20:02:30.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 msgr2=0x7f28a80b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f288c004a90 tx=0x7f28a000cb90 comp rx=0 tx=0).stop 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 shutdown_connections 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28a80cd6e0 0x7f28a80b0e50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f288806c980 0x7f288806ee40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f28a809d030 0x7f28a80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 --2- 172.21.15.88:0/979525986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f288c004ac0 0x7f28a80cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 >> 172.21.15.88:0/979525986 conn(0x7f28a80b2d50 msgr2=0x7f28a815d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 shutdown_connections 2024-04-26T20:02:30.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.361+0000 7f28ada00640 1 -- 172.21.15.88:0/979525986 wait complete. 2024-04-26T20:02:30.602 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:02:30.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.813+0000 7fb770800640 1 Processor -- start 2024-04-26T20:02:30.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.813+0000 7fb770800640 1 -- start start 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.814+0000 7fb770800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb76815fe50 0x7fb7681638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb770800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7680b5fb0 con 0x7fb7681598a0 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb770800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb768160370 con 0x7fb76806ce80 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb770800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb768159c80 con 0x7fb76815fe50 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb76de00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb7681598a0 0x7fb76806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35118/0 (socket says 172.21.15.88:35118) 2024-04-26T20:02:30.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb76de00640 1 -- 172.21.15.88:0/2315864143 learned_addr learned my addr 172.21.15.88:0/2315864143 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:30.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb76e800640 1 --2- 172.21.15.88:0/2315864143 >> v2:172.21.15.132:3300/0 conn(0x7fb76815fe50 0x7fb7681638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287144896 0 0) 0x7fb7680b5fb0 con 0x7fb7681598a0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.815+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb750003520 con 0x7fb7681598a0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2874605975 0 0) 0x7fb750003520 con 0x7fb7681598a0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 >> v2:172.21.15.132:3300/0 conn(0x7fb76815fe50 msgr2=0x7fb7681638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 --2- 172.21.15.88:0/2315864143 >> v2:172.21.15.132:3300/0 conn(0x7fb76815fe50 0x7fb7681638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 >> v1:172.21.15.132:6789/0 conn(0x7fb76806ce80 legacy=0x7fb76815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768162dd0 con 0x7fb7681598a0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb75c001670 con 0x7fb7681598a0 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 >> v1:172.21.15.88:6789/0 conn(0x7fb7681598a0 legacy=0x7fb76806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 0x7fb750003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb750004800 0x7fb750004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500052e0 0x7fb75000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb768159c80 con 0x7fb7500039b0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb75000c190 con 0x7fb7500052e0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb75000c390 con 0x7fb750004800 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76de00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 0x7fb750003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76e800640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500052e0 0x7fb75000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76de00640 1 -- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb750004800 msgr2=0x7fb750004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.816+0000 7fb76d400640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb750004800 0x7fb750004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76de00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb750004800 0x7fb750004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76de00640 1 -- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500052e0 msgr2=0x7fb75000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76de00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500052e0 0x7fb75000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76de00640 1 -- 172.21.15.88:0/2315864143 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb750003520 con 0x7fb7500039b0 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76e800640 1 -- 172.21.15.88:0/2315864143 reap_dead start 2024-04-26T20:02:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.817+0000 7fb76d400640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb750004800 0x7fb750004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb76de00640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 0x7fb750003db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c210 tx=0x7fb750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef0fcb502fd82f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c007d10 con 0x7fb7500039b0 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0058d0 con 0x7fb7500039b0 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb76ca00640 1 -- 172.21.15.88:0/2315864143 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c005d80 con 0x7fb7500039b0 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb770800640 1 -- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 msgr2=0x7fb750003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.818+0000 7fb770800640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 0x7fb750003db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c210 tx=0x7fb750005830 comp rx=0 tx=0).stop 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.819+0000 7fb770800640 1 -- 172.21.15.88:0/2315864143 shutdown_connections 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.819+0000 7fb770800640 1 --2- 172.21.15.88:0/2315864143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7500039b0 0x7fb750003db0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.819+0000 7fb770800640 1 -- 172.21.15.88:0/2315864143 >> 172.21.15.88:0/2315864143 conn(0x7fb7680b2d50 msgr2=0x7fb7680b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.819+0000 7fb770800640 1 -- 172.21.15.88:0/2315864143 shutdown_connections 2024-04-26T20:02:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.819+0000 7fb770800640 1 -- 172.21.15.88:0/2315864143 wait complete. 2024-04-26T20:02:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.820+0000 7fb770800640 1 Processor -- start 2024-04-26T20:02:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 -- start start 2024-04-26T20:02:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 0x7fb768146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb76809beb0 0x7fb76806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb768160650 con 0x7fb76809ba50 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb768163390 con 0x7fb7500039b0 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.821+0000 7fb770800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb7681638b0 con 0x7fb76809beb0 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58040/0 (socket says 172.21.15.88:58040) 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 -- 172.21.15.88:0/1282820138 learned_addr learned my addr 172.21.15.88:0/1282820138 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76e800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb76809beb0 0x7fb76806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76de00640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 0x7fb768146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 -- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb76809beb0 msgr2=0x7fb76806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb76809beb0 0x7fb76806f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 -- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 msgr2=0x7fb768146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 0x7fb768146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76d400640 1 -- 172.21.15.88:0/1282820138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7680cb370 con 0x7fb76809ba50 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.822+0000 7fb76de00640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 0x7fb768146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb76d400640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d490 tx=0x7fb75400d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754016030 con 0x7fb76809ba50 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754007020 con 0x7fb76809ba50 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7680cb650 con 0x7fb76809ba50 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754007410 con 0x7fb76809ba50 2024-04-26T20:02:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.823+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7680cbc10 con 0x7fb76809ba50 2024-04-26T20:02:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.824+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768069540 con 0x7fb76809ba50 2024-04-26T20:02:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.825+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb754019070 con 0x7fb76809ba50 2024-04-26T20:02:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.826+0000 7fb762000640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 0x7fb74806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.826+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fb75408b9b0 con 0x7fb76809ba50 2024-04-26T20:02:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.826+0000 7fb76de00640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 0x7fb74806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:30.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.827+0000 7fb76de00640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 0x7fb74806ee70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb75c033000 tx=0x7fb75c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:30.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:30.831+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb754059be0 con 0x7fb76809ba50 2024-04-26T20:02:31.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.070+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb768073fc0 con 0x7fb76809ba50 2024-04-26T20:02:31.071 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-04-26T20:02:31.070+0000 7fad57a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:02:31.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.074+0000 7fb762000640 1 -- 172.21.15.88:0/1282820138 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7fb7540594e0 con 0x7fb76809ba50 2024-04-26T20:02:31.075 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:02:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 msgr2=0x7fb74806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 0x7fb74806ee70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb75c033000 tx=0x7fb75c009030 comp rx=0 tx=0).stop 2024-04-26T20:02:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 msgr2=0x7fb76806f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d490 tx=0x7fb75400d950 comp rx=0 tx=0).stop 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 shutdown_connections 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb74806c9b0 0x7fb74806ee70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb76809beb0 0x7fb76806f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb76809ba50 0x7fb76806f060 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 --2- 172.21.15.88:0/1282820138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7500039b0 0x7fb768146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 >> 172.21.15.88:0/1282820138 conn(0x7fb7680b2d50 msgr2=0x7fb768147100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 shutdown_connections 2024-04-26T20:02:31.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.078+0000 7fb770800640 1 -- 172.21.15.88:0/1282820138 wait complete. 2024-04-26T20:02:31.092 INFO:tasks.ceph.mds.a.smithi088.stderr: -2735> 2024-04-26T20:02:06.063+0000 7f2acb200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:02:31.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:02:31.174 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:02:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.282+0000 7f379b600640 1 Processor -- start 2024-04-26T20:02:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.282+0000 7f379b600640 1 -- start start 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.283+0000 7f379b600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f379406ce80 0x7f37940b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.283+0000 7f379b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37940b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.283+0000 7f379b600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37940af800 con 0x7f379415d100 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.283+0000 7f379b600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f37940ae340 con 0x7f379406ce80 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.283+0000 7f379b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f37940afaa0 con 0x7f37940b5ca0 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f379406ce80 0x7f37940b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f379406ce80 0x7f37940b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58060/0 (socket says 172.21.15.88:58060) 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 -- 172.21.15.88:0/1887714373 learned_addr learned my addr 172.21.15.88:0/1887714373 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:31.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3799600640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37940b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 -- 172.21.15.88:0/1887714373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 msgr2=0x7f37940b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37940b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 -- 172.21.15.88:0/1887714373 >> v1:172.21.15.132:6789/0 conn(0x7f379415d100 legacy=0x7f379406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.284+0000 7f3793e00640 1 -- 172.21.15.88:0/1887714373 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37940afd80 con 0x7f379406ce80 2024-04-26T20:02:31.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.285+0000 7f3793e00640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f37940b3c30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3780009870 tx=0x7f378002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea79882ff7d5b086 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:31.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f3793400640 1 -- 172.21.15.88:0/1887714373 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780037070 con 0x7f379406ce80 2024-04-26T20:02:31.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f3793400640 1 -- 172.21.15.88:0/1887714373 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3780032440 con 0x7f379406ce80 2024-04-26T20:02:31.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f3793400640 1 -- 172.21.15.88:0/1887714373 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780032810 con 0x7f379406ce80 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 -- 172.21.15.88:0/1887714373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 msgr2=0x7f37940b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f37940b3c30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3780009870 tx=0x7f378002a210 comp rx=0 tx=0).stop 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 -- 172.21.15.88:0/1887714373 shutdown_connections 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37940b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 --2- 172.21.15.88:0/1887714373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f37940b3c30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.286+0000 7f379b600640 1 -- 172.21.15.88:0/1887714373 >> 172.21.15.88:0/1887714373 conn(0x7f37940b2d50 msgr2=0x7f37940b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.287+0000 7f379b600640 1 -- 172.21.15.88:0/1887714373 shutdown_connections 2024-04-26T20:02:31.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.287+0000 7f379b600640 1 -- 172.21.15.88:0/1887714373 wait complete. 2024-04-26T20:02:31.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.288+0000 7f379b600640 1 Processor -- start 2024-04-26T20:02:31.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.288+0000 7f379b600640 1 -- start start 2024-04-26T20:02:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37941543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f379415d100 0x7f37941548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f379409ee50 con 0x7f379406ce80 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f37940b0340 con 0x7f379415d100 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.289+0000 7f379b600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3794160b70 con 0x7f37940b5ca0 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58074/0 (socket says 172.21.15.88:58074) 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 -- 172.21.15.88:0/3246242563 learned_addr learned my addr 172.21.15.88:0/3246242563 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 -- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 msgr2=0x7f37941543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37941543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 -- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f379415d100 msgr2=0x7f37941548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:31.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f379415d100 0x7f37941548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 -- 172.21.15.88:0/3246242563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379409c830 con 0x7f379406ce80 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.290+0000 7f3798c00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f378800b420 tx=0x7f378800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.291+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788011830 con 0x7f379406ce80 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.291+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788016070 con 0x7f379406ce80 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.291+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788011c10 con 0x7f379406ce80 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.291+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379409cb10 con 0x7f379406ce80 2024-04-26T20:02:31.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.291+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379409d0d0 con 0x7f379406ce80 2024-04-26T20:02:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.292+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3794069650 con 0x7f379406ce80 2024-04-26T20:02:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.293+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f378801b460 con 0x7f379406ce80 2024-04-26T20:02:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.294+0000 7f3790c00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 0x7f377406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:31.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.294+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5795+0+0 (secure 0 0 0) 0x7f378808b720 con 0x7f379406ce80 2024-04-26T20:02:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.294+0000 7f3793e00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 0x7f377406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:31.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.294+0000 7f3793e00640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 0x7f377406ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3780034000 tx=0x7f378002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.298+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3788020020 con 0x7f379406ce80 2024-04-26T20:02:31.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:31.540+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3794073d00 con 0x7f379406ce80 2024-04-26T20:02:32.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.078+0000 7f3790c00640 1 -- 172.21.15.88:0/3246242563 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f3788059a90 con 0x7f379406ce80 2024-04-26T20:02:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 msgr2=0x7f377406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 0x7f377406ee40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3780034000 tx=0x7f378002c000 comp rx=0 tx=0).stop 2024-04-26T20:02:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 msgr2=0x7f3794149760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f378800b420 tx=0x7f378800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 shutdown_connections 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f379415d100 0x7f37941548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f377406c980 0x7f377406ee40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f37940b5ca0 0x7f37941543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 --2- 172.21.15.88:0/3246242563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f379406ce80 0x7f3794149760 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.082+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 >> 172.21.15.88:0/3246242563 conn(0x7f37940b2d50 msgr2=0x7f379406c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.083+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 shutdown_connections 2024-04-26T20:02:32.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.083+0000 7f379b600640 1 -- 172.21.15.88:0/3246242563 wait complete. 2024-04-26T20:02:32.105 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:02:32.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.309+0000 7f949c400640 1 Processor -- start 2024-04-26T20:02:32.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.310+0000 7f949c400640 1 -- start start 2024-04-26T20:02:32.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.311+0000 7f949c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 0x7f949415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.311+0000 7f949c400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f94940b3f50 0x7f94940b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.311+0000 7f949c400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949415c8d0 con 0x7f949406df80 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.311+0000 7f949c400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f94940b48e0 con 0x7f94940b3f50 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.311+0000 7f949c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f949415cbb0 con 0x7f949406f4a0 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9499000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f949406df80 0x7f949406f880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35128/0 (socket says 172.21.15.88:35128) 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9499000640 1 -- 172.21.15.88:0/771387677 learned_addr learned my addr 172.21.15.88:0/771387677 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:32.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f949a400640 1 --2- 172.21.15.88:0/771387677 >> v2:172.21.15.88:3300/0 conn(0x7f94940b3f50 0x7f94940b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9493e00640 1 -- 172.21.15.88:0/771387677 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083447736 0 0) 0x7f949415c8d0 con 0x7f949406df80 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9493e00640 1 -- 172.21.15.88:0/771387677 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f947c005730 con 0x7f949406df80 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f949a400640 1 -- 172.21.15.88:0/771387677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 msgr2=0x7f949415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9499a00640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 0x7f949415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f949a400640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 0x7f949415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f949a400640 1 -- 172.21.15.88:0/771387677 >> v1:172.21.15.88:6789/0 conn(0x7f949406df80 legacy=0x7f949406f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f949a400640 1 -- 172.21.15.88:0/771387677 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949415ce90 con 0x7f94940b3f50 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.312+0000 7f9499a00640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 0x7f949415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f949a400640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94940b3f50 0x7f94940b43a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f948c009d90 tx=0x7f948c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3843ea3d070e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f9493e00640 1 -- 172.21.15.88:0/771387677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c007740 con 0x7f94940b3f50 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f9493e00640 1 -- 172.21.15.88:0/771387677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948c0078e0 con 0x7f94940b3f50 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f9493e00640 1 -- 172.21.15.88:0/771387677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c007cb0 con 0x7f94940b3f50 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f949c400640 1 -- 172.21.15.88:0/771387677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94940b3f50 msgr2=0x7f94940b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.314+0000 7f949c400640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94940b3f50 0x7f94940b43a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f948c009d90 tx=0x7f948c02a660 comp rx=0 tx=0).stop 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 -- 172.21.15.88:0/771387677 shutdown_connections 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94940b3f50 0x7f94940b43a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 --2- 172.21.15.88:0/771387677 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406f4a0 0x7f949415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 -- 172.21.15.88:0/771387677 >> 172.21.15.88:0/771387677 conn(0x7f94940d0c80 msgr2=0x7f94940d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 -- 172.21.15.88:0/771387677 shutdown_connections 2024-04-26T20:02:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.315+0000 7f949c400640 1 -- 172.21.15.88:0/771387677 wait complete. 2024-04-26T20:02:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.316+0000 7f949c400640 1 Processor -- start 2024-04-26T20:02:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.316+0000 7f949c400640 1 -- start start 2024-04-26T20:02:32.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406df80 0x7f94940cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94940b3f50 0x7f94940cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94940ceb80 con 0x7f949406f4a0 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f949415d450 con 0x7f94940b3f50 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.317+0000 7f949c400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f94940ced00 con 0x7f949406df80 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58098/0 (socket says 172.21.15.88:58098) 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 -- 172.21.15.88:0/4122545935 learned_addr learned my addr 172.21.15.88:0/4122545935 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499a00640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406df80 0x7f94940cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f949a400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94940b3f50 0x7f94940cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 -- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406df80 msgr2=0x7f94940cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406df80 0x7f94940cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 -- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94940b3f50 msgr2=0x7f94940cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94940b3f50 0x7f94940cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.318+0000 7f9499000640 1 -- 172.21.15.88:0/4122545935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94940cc620 con 0x7f949406f4a0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f9499000640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480017070 con 0x7f949406f4a0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9480005ce0 con 0x7f949406f4a0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800072f0 con 0x7f949406f4a0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94940ca300 con 0x7f949406f4a0 2024-04-26T20:02:32.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.319+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94940ca8c0 con 0x7f949406f4a0 2024-04-26T20:02:32.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.320+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9494069540 con 0x7f949406f4a0 2024-04-26T20:02:32.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.321+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f948001a070 con 0x7f949406f4a0 2024-04-26T20:02:32.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.323+0000 7f9491600640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 0x7f947406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:32.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.323+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5795+0+0 (secure 0 0 0) 0x7f948008b120 con 0x7f949406f4a0 2024-04-26T20:02:32.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.323+0000 7f9499a00640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 0x7f947406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:32.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.324+0000 7f9499a00640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 0x7f947406ee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f948800a990 tx=0x7f9488008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:32.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.327+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9480059400 con 0x7f949406f4a0 2024-04-26T20:02:32.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:32.535+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94940caa90 con 0x7f949406f4a0 2024-04-26T20:02:33.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.082+0000 7f9491600640 1 -- 172.21.15.88:0/4122545935 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) ==== 181+0+0 (secure 0 0 0) 0x7f9480058d00 con 0x7f949406f4a0 2024-04-26T20:02:33.083 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:02:33.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 msgr2=0x7f947406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 0x7f947406ee70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f948800a990 tx=0x7f9488008040 comp rx=0 tx=0).stop 2024-04-26T20:02:33.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 msgr2=0x7f949415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f948000c430 tx=0x7f948000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 shutdown_connections 2024-04-26T20:02:33.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94940b3f50 0x7f94940cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f949406f4a0 0x7f949415e890 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f947406c9b0 0x7f947406ee70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 --2- 172.21.15.88:0/4122545935 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f949406df80 0x7f94940cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.087+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 >> 172.21.15.88:0/4122545935 conn(0x7f94940d0c80 msgr2=0x7f9494073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:33.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.088+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 shutdown_connections 2024-04-26T20:02:33.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.089+0000 7f949c400640 1 -- 172.21.15.88:0/4122545935 wait complete. 2024-04-26T20:02:33.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:02:33.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.320+0000 7ff38be00640 1 Processor -- start 2024-04-26T20:02:33.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.320+0000 7ff38be00640 1 -- start start 2024-04-26T20:02:33.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.321+0000 7ff38be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.321+0000 7ff38be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff38c0cb8b0 0x7ff38c0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff38c06df60 con 0x7ff38c143470 2024-04-26T20:02:33.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff38c146ba0 con 0x7ff38c0cb8b0 2024-04-26T20:02:33.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c06e200 con 0x7ff38c143850 2024-04-26T20:02:33.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff38c0cb8b0 0x7ff38c0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:50396/0 (socket says 172.21.15.88:50396) 2024-04-26T20:02:33.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.322+0000 7ff38a000640 1 -- 172.21.15.88:0/3703074385 learned_addr learned my addr 172.21.15.88:0/3703074385 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.323+0000 7ff38a000640 1 -- 172.21.15.88:0/3703074385 >> v2:172.21.15.132:3300/0 conn(0x7ff38c0cb8b0 msgr2=0x7ff38c0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.323+0000 7ff38a000640 1 --2- 172.21.15.88:0/3703074385 >> v2:172.21.15.132:3300/0 conn(0x7ff38c0cb8b0 0x7ff38c0723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.323+0000 7ff38a000640 1 -- 172.21.15.88:0/3703074385 >> v1:172.21.15.132:6789/0 conn(0x7ff38c143470 legacy=0x7ff38c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.323+0000 7ff38a000640 1 -- 172.21.15.88:0/3703074385 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c06e4e0 con 0x7ff38c143850 2024-04-26T20:02:33.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.324+0000 7ff38a000640 1 --2- 172.21.15.88:0/3703074385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f4b2b74631f3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.325+0000 7ff389600640 1 -- 172.21.15.88:0/3703074385 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378037070 con 0x7ff38c143850 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.325+0000 7ff389600640 1 -- 172.21.15.88:0/3703074385 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff378032440 con 0x7ff38c143850 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.325+0000 7ff389600640 1 -- 172.21.15.88:0/3703074385 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378032810 con 0x7ff38c143850 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 -- 172.21.15.88:0/3703074385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 msgr2=0x7ff38c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 --2- 172.21.15.88:0/3703074385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 -- 172.21.15.88:0/3703074385 shutdown_connections 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 --2- 172.21.15.88:0/3703074385 >> v2:172.21.15.132:3300/0 conn(0x7ff38c0cb8b0 0x7ff38c0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 --2- 172.21.15.88:0/3703074385 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c143850 0x7ff38c06fb10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 -- 172.21.15.88:0/3703074385 >> 172.21.15.88:0/3703074385 conn(0x7ff38c0d0b50 msgr2=0x7ff38c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 -- 172.21.15.88:0/3703074385 shutdown_connections 2024-04-26T20:02:33.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.326+0000 7ff38be00640 1 -- 172.21.15.88:0/3703074385 wait complete. 2024-04-26T20:02:33.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.328+0000 7ff38be00640 1 Processor -- start 2024-04-26T20:02:33.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.328+0000 7ff38be00640 1 -- start start 2024-04-26T20:02:33.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c0cb8b0 0x7ff38c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff38c143470 0x7ff38c144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0cbd00 con 0x7ff38c143850 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c070180 con 0x7ff38c143470 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c0cbe80 con 0x7ff38c0cb8b0 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58100/0 (socket says 172.21.15.88:58100) 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 -- 172.21.15.88:0/1648576458 learned_addr learned my addr 172.21.15.88:0/1648576458 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38a000640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff38c143470 0x7ff38c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38aa00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c0cb8b0 0x7ff38c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 -- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c0cb8b0 msgr2=0x7ff38c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c0cb8b0 0x7ff38c1f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 -- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff38c143470 msgr2=0x7ff38c144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff38c143470 0x7ff38c144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.329+0000 7ff38b400640 1 -- 172.21.15.88:0/1648576458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c1f5460 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff38b400640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff38400efc0 tx=0x7ff38400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384018020 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384014480 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384014870 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c1e3070 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.330+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c1e8640 con 0x7ff38c143850 2024-04-26T20:02:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.332+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c06d1f0 con 0x7ff38c143850 2024-04-26T20:02:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.332+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff384005d10 con 0x7ff38c143850 2024-04-26T20:02:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.333+0000 7ff382a00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 0x7ff36c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.333+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5357+0+0 (secure 0 0 0) 0x7ff38408c8e0 con 0x7ff38c143850 2024-04-26T20:02:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.334+0000 7ff38aa00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 0x7ff36c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:33.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.334+0000 7ff38aa00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 0x7ff36c06ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000f60 tx=0x7ff37c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:33.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.338+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff38c06d1f0 con 0x7ff38c143850 2024-04-26T20:02:33.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:33.536+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff38c0693c0 con 0x7ff38c143850 2024-04-26T20:02:34.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.086+0000 7ff382a00640 1 -- 172.21.15.88:0/1648576458 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) ==== 169+0+0 (secure 0 0 0) 0x7ff384022030 con 0x7ff38c143850 2024-04-26T20:02:34.087 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-04-26T20:02:34.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.089+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 msgr2=0x7ff36c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.089+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 0x7ff36c06ee40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000f60 tx=0x7ff37c009040 comp rx=0 tx=0).stop 2024-04-26T20:02:34.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.089+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 msgr2=0x7ff38c144ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.089+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff38400efc0 tx=0x7ff38400c490 comp rx=0 tx=0).stop 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 shutdown_connections 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff38c143850 0x7ff38c144ea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff38c143470 0x7ff38c144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff36c06c980 0x7ff36c06ee40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 --2- 172.21.15.88:0/1648576458 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff38c0cb8b0 0x7ff38c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 >> 172.21.15.88:0/1648576458 conn(0x7ff38c0d0b50 msgr2=0x7ff38c1424f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 shutdown_connections 2024-04-26T20:02:34.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.090+0000 7ff38be00640 1 -- 172.21.15.88:0/1648576458 wait complete. 2024-04-26T20:02:34.123 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:02:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.313+0000 7fb883a00640 1 Processor -- start 2024-04-26T20:02:34.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.313+0000 7fb883a00640 1 -- start start 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 0x7fb87c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 0x7fb87c06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb87c0add80 0x7fb87c072010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c0af800 con 0x7fb87c0add80 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c0ae340 con 0x7fb87c0b5ca0 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb883a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0afaa0 con 0x7fb87c15d100 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb881a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb87c0add80 0x7fb87c072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb881a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb87c0add80 0x7fb87c072010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58116/0 (socket says 172.21.15.88:58116) 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.315+0000 7fb881a00640 1 -- 172.21.15.88:0/1841134740 learned_addr learned my addr 172.21.15.88:0/1841134740 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb873e00640 1 --2- 172.21.15.88:0/1841134740 >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 0x7fb87c06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881000640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 0x7fb87c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881a00640 1 -- 172.21.15.88:0/1841134740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 msgr2=0x7fb87c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881a00640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 0x7fb87c06c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881a00640 1 -- 172.21.15.88:0/1841134740 >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 msgr2=0x7fb87c06ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881a00640 1 --2- 172.21.15.88:0/1841134740 >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 0x7fb87c06ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881a00640 1 -- 172.21.15.88:0/1841134740 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c0afd80 con 0x7fb87c0add80 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb881000640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 0x7fb87c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:34.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.316+0000 7fb873e00640 1 --2- 172.21.15.88:0/1841134740 >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 0x7fb87c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.317+0000 7fb881a00640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c072010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb87c075d50 tx=0x7fb87402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c53b9ef086a160 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.317+0000 7fb87be00640 1 -- 172.21.15.88:0/1841134740 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874036070 con 0x7fb87c0add80 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.317+0000 7fb87be00640 1 -- 172.21.15.88:0/1841134740 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb874031600 con 0x7fb87c0add80 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.317+0000 7fb87be00640 1 -- 172.21.15.88:0/1841134740 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8740319d0 con 0x7fb87c0add80 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 -- 172.21.15.88:0/1841134740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 msgr2=0x7fb87c072010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c072010 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb87c075d50 tx=0x7fb87402a610 comp rx=0 tx=0).stop 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 -- 172.21.15.88:0/1841134740 shutdown_connections 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c072010 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 --2- 172.21.15.88:0/1841134740 >> v2:172.21.15.132:3300/0 conn(0x7fb87c0b5ca0 0x7fb87c06ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 --2- 172.21.15.88:0/1841134740 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c15d100 0x7fb87c06c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 -- 172.21.15.88:0/1841134740 >> 172.21.15.88:0/1841134740 conn(0x7fb87c0b2d50 msgr2=0x7fb87c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 -- 172.21.15.88:0/1841134740 shutdown_connections 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.318+0000 7fb883a00640 1 -- 172.21.15.88:0/1841134740 wait complete. 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.320+0000 7fb883a00640 1 Processor -- start 2024-04-26T20:02:34.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.320+0000 7fb883a00640 1 -- start start 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c0b5ca0 0x7fb87c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb87c15d100 0x7fb87c09d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c09ee50 con 0x7fb87c0add80 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0b0340 con 0x7fb87c15d100 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb883a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c160b70 con 0x7fb87c0b5ca0 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58122/0 (socket says 172.21.15.88:58122) 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 -- 172.21.15.88:0/1811819542 learned_addr learned my addr 172.21.15.88:0/1811819542 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb87c15d100 0x7fb87c09d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 -- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c0b5ca0 msgr2=0x7fb87c14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c0b5ca0 0x7fb87c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 -- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb87c15d100 msgr2=0x7fb87c09d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.321+0000 7fb881000640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb87c15d100 0x7fb87c09d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb881000640 1 -- 172.21.15.88:0/1811819542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c151be0 con 0x7fb87c0add80 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb881000640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c7f0 tx=0x7fb86c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c015070 con 0x7fb87c0add80 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86c007820 con 0x7fb87c0add80 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c151e60 con 0x7fb87c0add80 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c007bd0 con 0x7fb87c0add80 2024-04-26T20:02:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.322+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c15ecf0 con 0x7fb87c0add80 2024-04-26T20:02:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.323+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c069650 con 0x7fb87c0add80 2024-04-26T20:02:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.324+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb86c018070 con 0x7fb87c0add80 2024-04-26T20:02:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.326+0000 7fb879600640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 0x7fb85806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.326+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fb86c08ba60 con 0x7fb87c0add80 2024-04-26T20:02:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.326+0000 7fb873e00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 0x7fb85806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.327+0000 7fb873e00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 0x7fb85806ee70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb864006fd0 tx=0x7fb864008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.330+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb87c151be0 con 0x7fb87c0add80 2024-04-26T20:02:34.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.544+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb87c073d00 con 0x7fb87c0add80 2024-04-26T20:02:34.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.549+0000 7fb879600640 1 -- 172.21.15.88:0/1811819542 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fb86c05a020 con 0x7fb87c0add80 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 msgr2=0x7fb85806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 0x7fb85806ee70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb864006fd0 tx=0x7fb864008040 comp rx=0 tx=0).stop 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 msgr2=0x7fb87c153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c7f0 tx=0x7fb86c00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 shutdown_connections 2024-04-26T20:02:34.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb87c15d100 0x7fb87c09d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb85806c9b0 0x7fb85806ee70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb87c0b5ca0 0x7fb87c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 --2- 172.21.15.88:0/1811819542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb87c0add80 0x7fb87c153940 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.551+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 >> 172.21.15.88:0/1811819542 conn(0x7fb87c0b2d50 msgr2=0x7fb87c0ae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.552+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 shutdown_connections 2024-04-26T20:02:34.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.552+0000 7fb883a00640 1 -- 172.21.15.88:0/1811819542 wait complete. 2024-04-26T20:02:34.570 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2024-04-26T20:02:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.748+0000 7fb043e00640 1 Processor -- start 2024-04-26T20:02:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.748+0000 7fb043e00640 1 -- start start 2024-04-26T20:02:34.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 0x7fb0440b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04415f5c0 con 0x7fb0440d3190 2024-04-26T20:02:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0440b8670 con 0x7fb0440d5b60 2024-04-26T20:02:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb0440b8950 con 0x7fb04415f1e0 2024-04-26T20:02:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb0440d3190 0x7fb0440b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35142/0 (socket says 172.21.15.88:35142) 2024-04-26T20:02:34.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.750+0000 7fb043400640 1 -- 172.21.15.88:0/3727813183 learned_addr learned my addr 172.21.15.88:0/3727813183 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb042a00640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 0x7fb0440b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436564234 0 0) 0x7fb04415f5c0 con 0x7fb0440d3190 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb024003520 con 0x7fb0440d3190 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 840372901 0 0) 0x7fb024003520 con 0x7fb0440d3190 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 msgr2=0x7fb0440b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 0x7fb0440b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 >> v1:172.21.15.132:6789/0 conn(0x7fb0440d5b60 legacy=0x7fb0440d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0440b8c30 con 0x7fb0440d3190 2024-04-26T20:02:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fb03c001670 con 0x7fb0440d3190 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 >> v1:172.21.15.88:6789/0 conn(0x7fb0440d3190 legacy=0x7fb0440b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 0x7fb0240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb024003fb0 0x7fb024004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb024004d40 0x7fb0240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0440b8950 con 0x7fb024004d40 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb02400bb00 con 0x7fb024003fb0 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.751+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb02400bd00 con 0x7fb04415f1e0 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb024004d40 0x7fb0240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb042000640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb024003fb0 0x7fb024004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 -- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 msgr2=0x7fb0240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415f1e0 0x7fb0240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 -- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb024003fb0 msgr2=0x7fb024004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:34.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb024003fb0 0x7fb024004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb043400640 1 -- 172.21.15.88:0/3727813183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb024003520 con 0x7fb024004d40 2024-04-26T20:02:34.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.752+0000 7fb042000640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb024003fb0 0x7fb024004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.753+0000 7fb043400640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb024004d40 0x7fb0240051c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c190 tx=0x7fb024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2211325ba5be0731 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.753+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c007c60 con 0x7fb024004d40 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.753+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c005ad0 con 0x7fb024004d40 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.753+0000 7fb041600640 1 -- 172.21.15.88:0/3727813183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c007070 con 0x7fb024004d40 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 -- 172.21.15.88:0/3727813183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb024004d40 msgr2=0x7fb0240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 --2- 172.21.15.88:0/3727813183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb024004d40 0x7fb0240051c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c190 tx=0x7fb024005830 comp rx=0 tx=0).stop 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043400640 1 -- 172.21.15.88:0/3727813183 reap_dead start 2024-04-26T20:02:34.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 -- 172.21.15.88:0/3727813183 shutdown_connections 2024-04-26T20:02:34.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 -- 172.21.15.88:0/3727813183 >> 172.21.15.88:0/3727813183 conn(0x7fb04409e2d0 msgr2=0x7fb0440d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:34.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 -- 172.21.15.88:0/3727813183 shutdown_connections 2024-04-26T20:02:34.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.754+0000 7fb043e00640 1 -- 172.21.15.88:0/3727813183 wait complete. 2024-04-26T20:02:34.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.756+0000 7fb043e00640 1 Processor -- start 2024-04-26T20:02:34.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.756+0000 7fb043e00640 1 -- start start 2024-04-26T20:02:34.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb044154660 0x7fb0440cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415ff80 0x7fb0440ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0440b7d90 con 0x7fb0440ce690 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb0440b91f0 con 0x7fb044154660 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb0440b7f10 con 0x7fb04415ff80 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58138/0 (socket says 172.21.15.88:58138) 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.757+0000 7fb043400640 1 -- 172.21.15.88:0/287595762 learned_addr learned my addr 172.21.15.88:0/287595762 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 -- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415ff80 msgr2=0x7fb0440ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415ff80 0x7fb0440ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 -- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb044154660 msgr2=0x7fb0440cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb044154660 0x7fb0440cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 -- 172.21.15.88:0/287595762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb044160720 con 0x7fb0440ce690 2024-04-26T20:02:34.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043400640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb03c02ad60 tx=0x7fb03c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c037040 con 0x7fb0440ce690 2024-04-26T20:02:34.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c041070 con 0x7fb0440ce690 2024-04-26T20:02:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c007c60 con 0x7fb0440ce690 2024-04-26T20:02:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.758+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb044160a00 con 0x7fb0440ce690 2024-04-26T20:02:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.759+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb044160fc0 con 0x7fb0440ce690 2024-04-26T20:02:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.760+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0440ceb10 con 0x7fb0440ce690 2024-04-26T20:02:34.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.761+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb03c0072b0 con 0x7fb0440ce690 2024-04-26T20:02:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.762+0000 7fb03aa00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 0x7fb02006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.762+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fb03c0ab770 con 0x7fb0440ce690 2024-04-26T20:02:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.762+0000 7fb042a00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 0x7fb02006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:34.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.763+0000 7fb042a00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 0x7fb02006ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb0440cee30 tx=0x7fb034009500 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:34.766+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb03c079db0 con 0x7fb0440ce690 2024-04-26T20:02:35.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.048+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) -- 0x7fb0440693c0 con 0x7fb0440ce690 2024-04-26T20:02:35.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.089+0000 7fb03aa00640 1 -- 172.21.15.88:0/287595762 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v180) ==== 133+0+0 (secure 0 0 0) 0x7fb03c0796b0 con 0x7fb0440ce690 2024-04-26T20:02:35.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 msgr2=0x7fb02006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 0x7fb02006ee40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb0440cee30 tx=0x7fb034009500 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 msgr2=0x7fb0440c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb03c02ad60 tx=0x7fb03c001970 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 shutdown_connections 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0440ce690 0x7fb0440c89c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb02006c980 0x7fb02006ee40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb04415ff80 0x7fb0440ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 --2- 172.21.15.88:0/287595762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb044154660 0x7fb0440cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.093+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 >> 172.21.15.88:0/287595762 conn(0x7fb04409e2d0 msgr2=0x7fb04406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.094+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 shutdown_connections 2024-04-26T20:02:35.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.094+0000 7fb043e00640 1 -- 172.21.15.88:0/287595762 wait complete. 2024-04-26T20:02:35.113 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2024-04-26T20:02:35.113 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2024-04-26T20:02:35.113 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:02:35.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.314+0000 7ffb7c200640 1 Processor -- start 2024-04-26T20:02:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.315+0000 7ffb7c200640 1 -- start start 2024-04-26T20:02:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.316+0000 7ffb7c200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb740add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.316+0000 7ffb7c200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb740cd650 0x7ffb7406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.316+0000 7ffb7c200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb740af9a0 con 0x7ffb740b4f50 2024-04-26T20:02:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.316+0000 7ffb7c200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ffb740ae2c0 con 0x7ffb740cd650 2024-04-26T20:02:35.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.316+0000 7ffb7c200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ffb740b05b0 con 0x7ffb7415cdf0 2024-04-26T20:02:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.317+0000 7ffb78e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb740cd650 0x7ffb7406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.317+0000 7ffb79800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb740add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.317+0000 7ffb78e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffb740cd650 0x7ffb7406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:58340/0 (socket says 172.21.15.88:58340) 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.317+0000 7ffb78e00640 1 -- 172.21.15.88:0/3620842667 learned_addr learned my addr 172.21.15.88:0/3620842667 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.317+0000 7ffb79800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb740add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:50450/0 (socket says 172.21.15.88:50450) 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.318+0000 7ffb78e00640 1 -- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 msgr2=0x7ffb740add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.318+0000 7ffb78e00640 1 --2- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb740add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.318+0000 7ffb78e00640 1 -- 172.21.15.88:0/3620842667 >> v1:172.21.15.132:6789/0 conn(0x7ffb740b4f50 legacy=0x7ffb74072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.318+0000 7ffb78e00640 1 -- 172.21.15.88:0/3620842667 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb740afc80 con 0x7ffb740cd650 2024-04-26T20:02:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.319+0000 7ffb78e00640 1 --2- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740cd650 0x7ffb7406c770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009870 tx=0x7ffb6002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f2a05cb83507ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb73e00640 1 -- 172.21.15.88:0/3620842667 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60037070 con 0x7ffb740cd650 2024-04-26T20:02:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb73e00640 1 -- 172.21.15.88:0/3620842667 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb60032440 con 0x7ffb740cd650 2024-04-26T20:02:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb73e00640 1 -- 172.21.15.88:0/3620842667 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60032810 con 0x7ffb740cd650 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 -- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740cd650 msgr2=0x7ffb7406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 --2- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740cd650 0x7ffb7406c770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009870 tx=0x7ffb6002a270 comp rx=0 tx=0).stop 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 -- 172.21.15.88:0/3620842667 shutdown_connections 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 --2- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740cd650 0x7ffb7406c770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 --2- 172.21.15.88:0/3620842667 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb740add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 -- 172.21.15.88:0/3620842667 >> 172.21.15.88:0/3620842667 conn(0x7ffb740b2d50 msgr2=0x7ffb740b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 -- 172.21.15.88:0/3620842667 shutdown_connections 2024-04-26T20:02:35.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.320+0000 7ffb7c200640 1 -- 172.21.15.88:0/3620842667 wait complete. 2024-04-26T20:02:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.322+0000 7ffb7c200640 1 Processor -- start 2024-04-26T20:02:35.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.322+0000 7ffb7c200640 1 -- start start 2024-04-26T20:02:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 0x7ffb7415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb7406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7415d1d0 con 0x7ffb740cd650 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ffb740b0240 con 0x7ffb740b4f50 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb7c200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7415d350 con 0x7ffb7415cdf0 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.323+0000 7ffb78e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58148/0 (socket says 172.21.15.88:58148) 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 -- 172.21.15.88:0/2037520769 learned_addr learned my addr 172.21.15.88:0/2037520769 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb79800640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 0x7ffb7415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb7a200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb7406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 -- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 msgr2=0x7ffb7406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb7406f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 -- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 msgr2=0x7ffb7415f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 0x7ffb7415f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 -- 172.21.15.88:0/2037520769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7415f000 con 0x7ffb740cd650 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb79800640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 0x7ffb7415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb7a200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb7406f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.324+0000 7ffb78e00640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a2b0 tx=0x7ffb600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.325+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60037040 con 0x7ffb740cd650 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.325+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb60042070 con 0x7ffb740cd650 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.325+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb741f89b0 con 0x7ffb740cd650 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.325+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60030860 con 0x7ffb740cd650 2024-04-26T20:02:35.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.325+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb741f9250 con 0x7ffb740cd650 2024-04-26T20:02:35.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.326+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb741f8da0 con 0x7ffb740cd650 2024-04-26T20:02:35.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.327+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ffb60030a00 con 0x7ffb740cd650 2024-04-26T20:02:35.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.328+0000 7ffb71600640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 0x7ffb5406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.328+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ffb600ac100 con 0x7ffb740cd650 2024-04-26T20:02:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.328+0000 7ffb79800640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 0x7ffb5406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.329+0000 7ffb79800640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 0x7ffb5406ee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009ac0 tx=0x7ffb68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.332+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffb7415f000 con 0x7ffb740cd650 2024-04-26T20:02:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.531+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb74069540 con 0x7ffb740cd650 2024-04-26T20:02:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.533+0000 7ffb71600640 1 -- 172.21.15.88:0/2037520769 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+13006 (secure 0 0 0) 0x7ffb6007a8d0 con 0x7ffb740cd650 2024-04-26T20:02:35.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:35.535 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":55,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:34.082438+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6837/649375356":"2024-04-27T20:02:31.071211+0000","172.21.15.88:6836/649375356":"2024-04-27T20:02:31.071211+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 msgr2=0x7ffb5406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 0x7ffb5406ee40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009ac0 tx=0x7ffb68008040 comp rx=0 tx=0).stop 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 msgr2=0x7ffb7414b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a2b0 tx=0x7ffb600046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 shutdown_connections 2024-04-26T20:02:35.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffb5406c980 0x7ffb5406ee40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffb7415cdf0 0x7ffb7406f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb740cd650 0x7ffb7414b1b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 --2- 172.21.15.88:0/2037520769 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb740b4f50 0x7ffb7415f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 >> 172.21.15.88:0/2037520769 conn(0x7ffb740b2d50 msgr2=0x7ffb740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 shutdown_connections 2024-04-26T20:02:35.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.536+0000 7ffb7c200640 1 -- 172.21.15.88:0/2037520769 wait complete. 2024-04-26T20:02:35.557 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:02:35.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:02:35.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.756+0000 7fa6f6600640 1 Processor -- start 2024-04-26T20:02:35.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.756+0000 7fa6f6600640 1 -- start start 2024-04-26T20:02:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6f00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f6600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6f00b4ed0 0x7fa6f00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f6600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f006e320 con 0x7fa6f0072260 2024-04-26T20:02:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f6600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f006e600 con 0x7fa6f00b4ed0 2024-04-26T20:02:35.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f006e8e0 con 0x7fa6f0071e80 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f5c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa6f0072260 0x7fa6f00726f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35152/0 (socket says 172.21.15.88:35152) 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f5c00640 1 -- 172.21.15.88:0/1968895092 learned_addr learned my addr 172.21.15.88:0/1968895092 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6f5200640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6f00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.758+0000 7fa6e7e00640 1 --2- 172.21.15.88:0/1968895092 >> v2:172.21.15.132:3300/0 conn(0x7fa6f00b4ed0 0x7fa6f00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763505687 0 0) 0x7fa6f006e320 con 0x7fa6f0072260 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d4003520 con 0x7fa6f0072260 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 311001462 0 0) 0x7fa6d4003520 con 0x7fa6f0072260 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 msgr2=0x7fa6f00b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6f00b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 >> v2:172.21.15.132:3300/0 conn(0x7fa6f00b4ed0 msgr2=0x7fa6f00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 --2- 172.21.15.88:0/1968895092 >> v2:172.21.15.132:3300/0 conn(0x7fa6f00b4ed0 0x7fa6f00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f006ebc0 con 0x7fa6f0072260 2024-04-26T20:02:35.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa6e80019f0 con 0x7fa6f0072260 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 >> v1:172.21.15.88:6789/0 conn(0x7fa6f0072260 legacy=0x7fa6f00726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6d4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6d4003fa0 0x7fa6d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006e600 con 0x7fa6d4004d40 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006e8e0 con 0x7fa6d4003fa0 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.759+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d400b9f0 con 0x7fa6f0071e80 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6e7e00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6d4003fa0 0x7fa6d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5200640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6d4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 -- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 msgr2=0x7fa6d4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6d4003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 -- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6d4003fa0 msgr2=0x7fa6d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:35.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6d4003fa0 0x7fa6d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5c00640 1 -- 172.21.15.88:0/1968895092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d4003520 con 0x7fa6d4004d40 2024-04-26T20:02:35.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.760+0000 7fa6f5200640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0071e80 0x7fa6d4003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.761+0000 7fa6f5c00640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6d40051c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800a920 tx=0x7fa6d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2811adcf210dd73c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.761+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800a4c0 con 0x7fa6d4004d40 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.761+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e802f070 con 0x7fa6d4004d40 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.761+0000 7fa6efe00640 1 -- 172.21.15.88:0/1968895092 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e802bd40 con 0x7fa6d4004d40 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 -- 172.21.15.88:0/1968895092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 msgr2=0x7fa6d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 --2- 172.21.15.88:0/1968895092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6d40051c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800a920 tx=0x7fa6d4005830 comp rx=0 tx=0).stop 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f5c00640 1 -- 172.21.15.88:0/1968895092 reap_dead start 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 -- 172.21.15.88:0/1968895092 shutdown_connections 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 -- 172.21.15.88:0/1968895092 >> 172.21.15.88:0/1968895092 conn(0x7fa6f00b2900 msgr2=0x7fa6f00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 -- 172.21.15.88:0/1968895092 shutdown_connections 2024-04-26T20:02:35.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.762+0000 7fa6f6600640 1 -- 172.21.15.88:0/1968895092 wait complete. 2024-04-26T20:02:35.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.764+0000 7fa6f6600640 1 Processor -- start 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.764+0000 7fa6f6600640 1 -- start start 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0159f40 0x7fa6f015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f01520f0 0x7fa6f0152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00b3ee0 con 0x7fa6d4004d40 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00b5ff0 con 0x7fa6f01520f0 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f00b4060 con 0x7fa6f0159f40 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58176/0 (socket says 172.21.15.88:58176) 2024-04-26T20:02:35.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 -- 172.21.15.88:0/988867777 learned_addr learned my addr 172.21.15.88:0/988867777 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 -- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0159f40 msgr2=0x7fa6f015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0159f40 0x7fa6f015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 -- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f01520f0 msgr2=0x7fa6f0152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.765+0000 7fa6f5200640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f01520f0 0x7fa6f0152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6f5200640 1 -- 172.21.15.88:0/988867777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f015f880 con 0x7fa6d4004d40 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6f5200640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00108b0 tx=0x7fa6e0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000f040 con 0x7fa6d4004d40 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e000ea80 con 0x7fa6d4004d40 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000ee30 con 0x7fa6d4004d40 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f015fb60 con 0x7fa6d4004d40 2024-04-26T20:02:35.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.766+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f0152ab0 con 0x7fa6d4004d40 2024-04-26T20:02:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.767+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f00693c0 con 0x7fa6d4004d40 2024-04-26T20:02:35.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.768+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa6e001c400 con 0x7fa6d4004d40 2024-04-26T20:02:35.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.769+0000 7fa6ed600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 0x7fa6cc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:35.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.769+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fa6e008c050 con 0x7fa6d4004d40 2024-04-26T20:02:35.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.770+0000 7fa6e7e00640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 0x7fa6cc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:35.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.770+0000 7fa6e7e00640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 0x7fa6cc06ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6f01550a0 tx=0x7fa6d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:35.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.774+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6e0018030 con 0x7fa6d4004d40 2024-04-26T20:02:35.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.959+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6f015e3e0 con 0x7fa6d4004d40 2024-04-26T20:02:35.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.960+0000 7fa6ed600640 1 -- 172.21.15.88:0/988867777 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7fa6e005a600 con 0x7fa6d4004d40 2024-04-26T20:02:35.961 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:02:35.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 msgr2=0x7fa6cc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 0x7fa6cc06ee40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6f01550a0 tx=0x7fa6d8008040 comp rx=0 tx=0).stop 2024-04-26T20:02:35.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 msgr2=0x7fa6f0148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:35.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00108b0 tx=0x7fa6e0010d70 comp rx=0 tx=0).stop 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 shutdown_connections 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.962+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6f01520f0 0x7fa6f0152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6cc06c980 0x7fa6cc06ee40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6f0159f40 0x7fa6f015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 --2- 172.21.15.88:0/988867777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4004d40 0x7fa6f0148720 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 >> 172.21.15.88:0/988867777 conn(0x7fa6f00b2900 msgr2=0x7fa6f00ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 shutdown_connections 2024-04-26T20:02:35.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:35.963+0000 7fa6f6600640 1 -- 172.21.15.88:0/988867777 wait complete. 2024-04-26T20:02:35.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2024-04-26T20:02:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.179+0000 7ff0f3a00640 1 Processor -- start 2024-04-26T20:02:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.179+0000 7ff0f3a00640 1 -- start start 2024-04-26T20:02:36.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.180+0000 7ff0f3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0ec0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f3a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff0ec06ce80 0x7ff0ec0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f3a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec0af9a0 con 0x7ff0ec0cd650 2024-04-26T20:02:36.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f3a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ec0ae2c0 con 0x7ff0ec06ce80 2024-04-26T20:02:36.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b05b0 con 0x7ff0ec15cdf0 2024-04-26T20:02:36.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0e3e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff0ec0cd650 0x7ff0ec06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35162/0 (socket says 172.21.15.88:35162) 2024-04-26T20:02:36.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0e3e00640 1 -- 172.21.15.88:0/2379200485 learned_addr learned my addr 172.21.15.88:0/2379200485 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704144423 0 0) 0x7ff0ec0af9a0 con 0x7ff0ec0cd650 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f1a00640 1 --2- 172.21.15.88:0/2379200485 >> v2:172.21.15.132:3300/0 conn(0x7ff0ec06ce80 0x7ff0ec0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0f1000640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0ec0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.181+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0d0003520 con 0x7ff0ec0cd650 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1012939088 0 0) 0x7ff0d0003520 con 0x7ff0ec0cd650 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 msgr2=0x7ff0ec0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0ec0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 >> v2:172.21.15.132:3300/0 conn(0x7ff0ec06ce80 msgr2=0x7ff0ec0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 --2- 172.21.15.88:0/2379200485 >> v2:172.21.15.132:3300/0 conn(0x7ff0ec06ce80 0x7ff0ec0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec0afc80 con 0x7ff0ec0cd650 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7ff0d4001670 con 0x7ff0ec0cd650 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0f1a00640 1 --2- 172.21.15.88:0/2379200485 >> v2:172.21.15.132:3300/0 conn(0x7ff0ec06ce80 0x7ff0ec0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:02:36.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 >> v1:172.21.15.88:6789/0 conn(0x7ff0ec0cd650 legacy=0x7ff0ec06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d00039b0 0x7ff0d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d0004800 0x7ff0d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b05b0 con 0x7ff0d00039b0 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d000b9f0 con 0x7ff0d0004800 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d000bbf0 con 0x7ff0ec15cdf0 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.182+0000 7ff0f1000640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d00039b0 0x7ff0d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1a00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0e3e00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d0004800 0x7ff0d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1000640 1 -- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 msgr2=0x7ff0d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1000640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec15cdf0 0x7ff0d00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1000640 1 -- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d0004800 msgr2=0x7ff0d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1000640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d0004800 0x7ff0d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.183+0000 7ff0f1000640 1 -- 172.21.15.88:0/2379200485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d0003520 con 0x7ff0d00039b0 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f1000640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d00039b0 0x7ff0d0003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00ba00 tx=0x7ff0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f20f6c15df0e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc030070 con 0x7ff0d00039b0 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc02ad80 con 0x7ff0d00039b0 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0ebe00640 1 -- 172.21.15.88:0/2379200485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00b600 con 0x7ff0d00039b0 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f3a00640 1 -- 172.21.15.88:0/2379200485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d00039b0 msgr2=0x7ff0d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/2379200485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d00039b0 0x7ff0d0003dd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00ba00 tx=0x7ff0d0005830 comp rx=0 tx=0).stop 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f1a00640 1 -- 172.21.15.88:0/2379200485 reap_dead start 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f3a00640 1 -- 172.21.15.88:0/2379200485 shutdown_connections 2024-04-26T20:02:36.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.184+0000 7ff0f3a00640 1 -- 172.21.15.88:0/2379200485 >> 172.21.15.88:0/2379200485 conn(0x7ff0ec0b2d50 msgr2=0x7ff0ec0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.185+0000 7ff0f3a00640 1 -- 172.21.15.88:0/2379200485 shutdown_connections 2024-04-26T20:02:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.185+0000 7ff0f3a00640 1 -- 172.21.15.88:0/2379200485 wait complete. 2024-04-26T20:02:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.186+0000 7ff0f3a00640 1 Processor -- start 2024-04-26T20:02:36.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.186+0000 7ff0f3a00640 1 -- start start 2024-04-26T20:02:36.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d00039b0 0x7ff0ec15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec147190 0x7ff0ec15fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec15d1d0 con 0x7ff0ec147830 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0b0240 con 0x7ff0d00039b0 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0f3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec15d350 con 0x7ff0ec147190 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0e3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0e3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58194/0 (socket says 172.21.15.88:58194) 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.187+0000 7ff0e3e00640 1 -- 172.21.15.88:0/3127873955 learned_addr learned my addr 172.21.15.88:0/3127873955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 -- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec147190 msgr2=0x7ff0ec15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec147190 0x7ff0ec15fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 -- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d00039b0 msgr2=0x7ff0ec15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d00039b0 0x7ff0ec15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 -- 172.21.15.88:0/3127873955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec160140 con 0x7ff0ec147830 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e3e00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400d840 tx=0x7ff0d400dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:36.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4019070 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d40074e0 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4004a20 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.188+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec14d880 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.189+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec14de40 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.190+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec069540 con 0x7ff0ec147830 2024-04-26T20:02:36.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.191+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff0d4004bc0 con 0x7ff0ec147830 2024-04-26T20:02:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.191+0000 7ff0e9600640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 0x7ff0c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.192+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ff0d408b460 con 0x7ff0ec147830 2024-04-26T20:02:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.192+0000 7ff0f1000640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 0x7ff0c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:36.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.192+0000 7ff0f1000640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 0x7ff0c806ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00057e0 tx=0x7ff0dc02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:36.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.196+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0d4059990 con 0x7ff0ec147830 2024-04-26T20:02:36.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:36.448+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) -- 0x7ff0ec09d450 con 0x7ff0ec147830 2024-04-26T20:02:37.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.102+0000 7ff0e9600640 1 -- 172.21.15.88:0/3127873955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v182) ==== 161+0+0 (secure 0 0 0) 0x7ff0d4059290 con 0x7ff0ec147830 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 msgr2=0x7ff0c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 0x7ff0c806ee40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00057e0 tx=0x7ff0dc02c000 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 msgr2=0x7ff0ec15f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400d840 tx=0x7ff0d400dd00 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 shutdown_connections 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff0c806c980 0x7ff0c806ee40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0ec147190 0x7ff0ec15fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0ec147830 0x7ff0ec15f6c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 --2- 172.21.15.88:0/3127873955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0d00039b0 0x7ff0ec15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 >> 172.21.15.88:0/3127873955 conn(0x7ff0ec0b2d50 msgr2=0x7ff0ec15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 shutdown_connections 2024-04-26T20:02:37.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.106+0000 7ff0f3a00640 1 -- 172.21.15.88:0/3127873955 wait complete. 2024-04-26T20:02:37.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:02:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.333+0000 7ffaa7200640 1 Processor -- start 2024-04-26T20:02:37.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.333+0000 7ffaa7200640 1 -- start start 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa7200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ffaa80b4ed0 0x7ffaa80b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa7200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ffaa8072970 0x7ffaa80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa7200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa806e320 con 0x7ffaa8071e80 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa7200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa806e600 con 0x7ffaa8072970 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa7200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa806e8e0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa6800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ffaa8072970 0x7ffaa80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa6800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ffaa8072970 0x7ffaa80ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58196/0 (socket says 172.21.15.88:58196) 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa5e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ffaa8071e80 0x7ffaa8072260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35176/0 (socket says 172.21.15.88:35176) 2024-04-26T20:02:37.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.335+0000 7ffaa6800640 1 -- 172.21.15.88:0/3252223478 learned_addr learned my addr 172.21.15.88:0/3252223478 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffa9d400640 1 --2- 172.21.15.88:0/3252223478 >> v2:172.21.15.132:3300/0 conn(0x7ffaa80b4ed0 0x7ffaa80b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffaa6800640 1 -- 172.21.15.88:0/3252223478 >> v2:172.21.15.132:3300/0 conn(0x7ffaa80b4ed0 msgr2=0x7ffaa80b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffaa6800640 1 --2- 172.21.15.88:0/3252223478 >> v2:172.21.15.132:3300/0 conn(0x7ffaa80b4ed0 0x7ffaa80b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffaa6800640 1 -- 172.21.15.88:0/3252223478 >> v1:172.21.15.88:6789/0 conn(0x7ffaa8071e80 legacy=0x7ffaa8072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffaa6800640 1 -- 172.21.15.88:0/3252223478 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa806ebc0 con 0x7ffaa8072970 2024-04-26T20:02:37.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.336+0000 7ffaa5400640 1 -- 172.21.15.88:0/3252223478 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718368899 0 0) 0x7ffaa806e320 con 0x7ffaa8071e80 2024-04-26T20:02:37.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.337+0000 7ffaa6800640 1 --2- 172.21.15.88:0/3252223478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa8072970 0x7ffaa80ae8c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80cce70 tx=0x7ffaa002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d8626c16ed7311 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:37.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.337+0000 7ffaa5400640 1 -- 172.21.15.88:0/3252223478 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0036070 con 0x7ffaa8072970 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.337+0000 7ffaa5400640 1 -- 172.21.15.88:0/3252223478 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa0031600 con 0x7ffaa8072970 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.337+0000 7ffaa5400640 1 -- 172.21.15.88:0/3252223478 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00319d0 con 0x7ffaa8072970 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 -- 172.21.15.88:0/3252223478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa8072970 msgr2=0x7ffaa80ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 --2- 172.21.15.88:0/3252223478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa8072970 0x7ffaa80ae8c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80cce70 tx=0x7ffaa002a730 comp rx=0 tx=0).stop 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 -- 172.21.15.88:0/3252223478 shutdown_connections 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 --2- 172.21.15.88:0/3252223478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa8072970 0x7ffaa80ae8c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 --2- 172.21.15.88:0/3252223478 >> v2:172.21.15.132:3300/0 conn(0x7ffaa80b4ed0 0x7ffaa80b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 -- 172.21.15.88:0/3252223478 >> 172.21.15.88:0/3252223478 conn(0x7ffaa80b2900 msgr2=0x7ffaa80b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 -- 172.21.15.88:0/3252223478 shutdown_connections 2024-04-26T20:02:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.338+0000 7ffaa7200640 1 -- 172.21.15.88:0/3252223478 wait complete. 2024-04-26T20:02:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.340+0000 7ffaa7200640 1 Processor -- start 2024-04-26T20:02:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.340+0000 7ffaa7200640 1 -- start start 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa8071e80 0x7ffaa815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaa8072970 0x7ffaa81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa806f0e0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80b5ff0 con 0x7ffaa8071e80 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa7200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa806f260 con 0x7ffaa8072970 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58206/0 (socket says 172.21.15.88:58206) 2024-04-26T20:02:37.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.341+0000 7ffaa6800640 1 -- 172.21.15.88:0/1455189879 learned_addr learned my addr 172.21.15.88:0/1455189879 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa5e00640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa8071e80 0x7ffaa815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 -- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaa8072970 msgr2=0x7ffaa81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaa8072970 0x7ffaa81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 -- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa8071e80 msgr2=0x7ffaa815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa8071e80 0x7ffaa815ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 -- 172.21.15.88:0/1455189879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa815b470 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa6800640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009f40 tx=0x7ffaa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0036050 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa0042070 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0009510 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa815b750 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.342+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa815bff0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.344+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa80693c0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.345+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ffaa0007bf0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.345+0000 7ffa9ea00640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 0x7ffa8406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.346+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ffaa00ac800 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.346+0000 7ffaa5e00640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 0x7ffa8406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.346+0000 7ffaa5e00640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 0x7ffa8406ee40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:37.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.350+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffaa007adc0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.548+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffaa815e520 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.556+0000 7ffa9ea00640 1 -- 172.21.15.88:0/1455189879 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7ffaa007a6c0 con 0x7ffaa80b4ed0 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 msgr2=0x7ffa8406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 0x7ffa8406ee40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa98009040 comp rx=0 tx=0).stop 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 msgr2=0x7ffaa8149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009f40 tx=0x7ffaa0004a40 comp rx=0 tx=0).stop 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 shutdown_connections 2024-04-26T20:02:37.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa80b4ed0 0x7ffaa8149000 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ffa8406c980 0x7ffa8406ee40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ffaa8072970 0x7ffaa81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 --2- 172.21.15.88:0/1455189879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa8071e80 0x7ffaa815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.559+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 >> 172.21.15.88:0/1455189879 conn(0x7ffaa80b2900 msgr2=0x7ffaa80aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.560+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 shutdown_connections 2024-04-26T20:02:37.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:37.560+0000 7ffaa7200640 1 -- 172.21.15.88:0/1455189879 wait complete. 2024-04-26T20:02:37.578 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:02:37.578 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:02:37.684 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:02:37.790 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:02:37.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:37.791 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:37.846 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:02:37.847 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:02:37.847 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:37.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:02:37.904 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:37.904 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:37.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:02:37.960 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:02:37.960 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:38.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:02:38.015 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:02:38.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:38.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.218+0000 7ff36ac00640 1 Processor -- start 2024-04-26T20:02:38.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.218+0000 7ff36ac00640 1 -- start start 2024-04-26T20:02:38.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff36ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36415cdf0 0x7ff3640add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff36ac00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff3640cd650 0x7ff36406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff36ac00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3640af9a0 con 0x7ff3640b4f50 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff36ac00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff3640ae2c0 con 0x7ff3640cd650 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff36ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff3640b05b0 con 0x7ff36415cdf0 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff363400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff3640cd650 0x7ff36406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff363400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff3640cd650 0x7ff36406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58220/0 (socket says 172.21.15.88:58220) 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff363400640 1 -- 172.21.15.88:0/3410422661 learned_addr learned my addr 172.21.15.88:0/3410422661 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff368c00640 1 --1- 172.21.15.88:0/3410422661 >> v1:172.21.15.88:6789/0 conn(0x7ff3640b4f50 0x7ff364072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:35182/0 (socket says 172.21.15.88:35182) 2024-04-26T20:02:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.220+0000 7ff363e00640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36415cdf0 0x7ff3640add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.221+0000 7ff363400640 1 -- 172.21.15.88:0/3410422661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36415cdf0 msgr2=0x7ff3640add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.221+0000 7ff363400640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36415cdf0 0x7ff3640add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.221+0000 7ff363400640 1 -- 172.21.15.88:0/3410422661 >> v1:172.21.15.88:6789/0 conn(0x7ff3640b4f50 legacy=0x7ff364072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.221+0000 7ff363400640 1 -- 172.21.15.88:0/3410422661 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640afc80 con 0x7ff3640cd650 2024-04-26T20:02:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.221+0000 7ff362a00640 1 -- 172.21.15.88:0/3410422661 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343259315 0 0) 0x7ff3640af9a0 con 0x7ff3640b4f50 2024-04-26T20:02:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.222+0000 7ff363400640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640cd650 0x7ff36406c770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e50 tx=0x7ff35002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaf79cfec126a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.222+0000 7ff362a00640 1 -- 172.21.15.88:0/3410422661 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350036070 con 0x7ff3640cd650 2024-04-26T20:02:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.222+0000 7ff362a00640 1 -- 172.21.15.88:0/3410422661 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff350031600 con 0x7ff3640cd650 2024-04-26T20:02:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.222+0000 7ff362a00640 1 -- 172.21.15.88:0/3410422661 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3500319d0 con 0x7ff3640cd650 2024-04-26T20:02:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 -- 172.21.15.88:0/3410422661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640cd650 msgr2=0x7ff36406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640cd650 0x7ff36406c770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff350009e50 tx=0x7ff35002a710 comp rx=0 tx=0).stop 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 -- 172.21.15.88:0/3410422661 shutdown_connections 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640cd650 0x7ff36406c770 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3410422661 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff36415cdf0 0x7ff3640add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 -- 172.21.15.88:0/3410422661 >> 172.21.15.88:0/3410422661 conn(0x7ff3640b2d50 msgr2=0x7ff3640b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 -- 172.21.15.88:0/3410422661 shutdown_connections 2024-04-26T20:02:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.223+0000 7ff36ac00640 1 -- 172.21.15.88:0/3410422661 wait complete. 2024-04-26T20:02:38.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.225+0000 7ff36ac00640 1 Processor -- start 2024-04-26T20:02:38.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.225+0000 7ff36ac00640 1 -- start start 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.225+0000 7ff36ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff36ac00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3640cd650 0x7ff3640cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff36ac00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff36415cdf0 0x7ff3640cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff36ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff36415d1d0 con 0x7ff3640b4f50 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff36ac00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff3640b0240 con 0x7ff36415cdf0 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff36ac00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff36415d350 con 0x7ff3640cd650 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58228/0 (socket says 172.21.15.88:58228) 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 -- 172.21.15.88:0/3261777950 learned_addr learned my addr 172.21.15.88:0/3261777950 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 -- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3640cd650 msgr2=0x7ff3640cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3640cd650 0x7ff3640cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 -- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff36415cdf0 msgr2=0x7ff3640cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff36415cdf0 0x7ff3640cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.226+0000 7ff363e00640 1 -- 172.21.15.88:0/3261777950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640cf790 con 0x7ff3640b4f50 2024-04-26T20:02:38.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff363e00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff35400d830 tx=0x7ff35400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35400b6c0 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff354019070 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3640cfa70 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35400ba80 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.227+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3640d0030 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.228+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff364069540 con 0x7ff3640b4f50 2024-04-26T20:02:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.229+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff35400bc20 con 0x7ff3640b4f50 2024-04-26T20:02:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.230+0000 7ff35be00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 0x7ff34406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.230+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ff35408be50 con 0x7ff3640b4f50 2024-04-26T20:02:38.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.230+0000 7ff363400640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 0x7ff34406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.231+0000 7ff363400640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 0x7ff34406ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff350033000 tx=0x7ff35002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.234+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff35401c360 con 0x7ff3640b4f50 2024-04-26T20:02:38.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.475+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3640b34b0 con 0x7ff3640b4f50 2024-04-26T20:02:38.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.476+0000 7ff35be00640 1 -- 172.21.15.88:0/3261777950 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+3869 (secure 0 0 0) 0x7ff35405a490 con 0x7ff3640b4f50 2024-04-26T20:02:38.477 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:38.477 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/2082385109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":2082385109},{"type":"v1","addr":"172.21.15.132:6833","nonce":2082385109}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":4960,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/284333135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":284333135},{"type":"v1","addr":"172.21.15.132:6835","nonce":284333135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5697,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/4073325422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":4073325422},{"type":"v1","addr":"172.21.15.88:6835","nonce":4073325422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":6066,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/376734234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":376734234},{"type":"v1","addr":"172.21.15.88:6837","nonce":376734234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[]} 2024-04-26T20:02:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 msgr2=0x7ff34406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 0x7ff34406ee40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff350033000 tx=0x7ff35002e000 comp rx=0 tx=0).stop 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 msgr2=0x7ff3640b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff35400d830 tx=0x7ff35400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 shutdown_connections 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff36415cdf0 0x7ff3640cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff34406c980 0x7ff34406ee40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff3640cd650 0x7ff3640cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 --2- 172.21.15.88:0/3261777950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3640b4f50 0x7ff3640b1ba0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.478+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 >> 172.21.15.88:0/3261777950 conn(0x7ff3640b2d50 msgr2=0x7ff36415b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.479+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 shutdown_connections 2024-04-26T20:02:38.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.479+0000 7ff36ac00640 1 -- 172.21.15.88:0/3261777950 wait complete. 2024-04-26T20:02:38.499 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:02:38.499 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:02:38.499 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:02:38.500 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:02:38.500 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:02:38.500 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:02:38.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:02:38.500 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:02:38.501 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:02:38.501 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:02:38.501 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:02:38.501 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:02:38.501 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:02:38.501 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:02:38.502 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:02:38.502 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:02:38.516 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:02:38.517 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:02:38.519 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:02:38.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:02:38.526 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:02:38.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:02:38.528 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:02:38.528 DEBUG:teuthology.orchestra.run.smithi132:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:02:38.565 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:02:38.566 DEBUG:teuthology.parallel:result is None 2024-04-26T20:02:38.572 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:02:38.572 DEBUG:teuthology.parallel:result is None 2024-04-26T20:02:38.576 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:02:38.576 DEBUG:teuthology.parallel:result is None 2024-04-26T20:02:38.582 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:02:38.582 DEBUG:teuthology.parallel:result is None 2024-04-26T20:02:38.583 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:02:38.706 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-04-26T20:02:38.711 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-04-26T20:02:38.736 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-26T20:02:38.736 INFO:tasks.ceph.mds.b.smithi132.stdout:starting mds.b at 2024-04-26T20:02:38.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.754+0000 7fda2b200640 1 Processor -- start 2024-04-26T20:02:38.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.754+0000 7fda2b200640 1 -- start start 2024-04-26T20:02:38.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fda2c15f1b0 0x7fda2c0d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda2c162b00 0x7fda2c162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c163910 con 0x7fda2c162b00 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fda2c0d3f50 con 0x7fda2c15f1b0 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda2b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c0d36b0 con 0x7fda2c163480 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda29400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda2c162b00 0x7fda2c162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda29400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fda2c162b00 0x7fda2c162f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58262/0 (socket says 172.21.15.88:58262) 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.755+0000 7fda29400640 1 -- 172.21.15.88:0/3759000778 learned_addr learned my addr 172.21.15.88:0/3759000778 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29e00640 1 --2- 172.21.15.88:0/3759000778 >> v2:172.21.15.132:3300/0 conn(0x7fda2c15f1b0 0x7fda2c0d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda2a800640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29400640 1 -- 172.21.15.88:0/3759000778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 msgr2=0x7fda2c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29400640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29400640 1 -- 172.21.15.88:0/3759000778 >> v2:172.21.15.132:3300/0 conn(0x7fda2c15f1b0 msgr2=0x7fda2c0d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29400640 1 --2- 172.21.15.88:0/3759000778 >> v2:172.21.15.132:3300/0 conn(0x7fda2c15f1b0 0x7fda2c0d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda29400640 1 -- 172.21.15.88:0/3759000778 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c0d3990 con 0x7fda2c162b00 2024-04-26T20:02:38.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.756+0000 7fda2a800640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.757+0000 7fda29400640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c162f40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda2c06f570 tx=0x7fda1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196ceee617bd1c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.757+0000 7fda28a00640 1 -- 172.21.15.88:0/3759000778 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18037070 con 0x7fda2c162b00 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.757+0000 7fda28a00640 1 -- 172.21.15.88:0/3759000778 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda18032440 con 0x7fda2c162b00 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.757+0000 7fda28a00640 1 -- 172.21.15.88:0/3759000778 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18032810 con 0x7fda2c162b00 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 -- 172.21.15.88:0/3759000778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 msgr2=0x7fda2c162f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c162f40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda2c06f570 tx=0x7fda1802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 -- 172.21.15.88:0/3759000778 shutdown_connections 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 --2- 172.21.15.88:0/3759000778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c162f40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 --2- 172.21.15.88:0/3759000778 >> v2:172.21.15.132:3300/0 conn(0x7fda2c15f1b0 0x7fda2c0d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 -- 172.21.15.88:0/3759000778 >> 172.21.15.88:0/3759000778 conn(0x7fda2c09e230 msgr2=0x7fda2c0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:38.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 -- 172.21.15.88:0/3759000778 shutdown_connections 2024-04-26T20:02:38.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.758+0000 7fda2b200640 1 -- 172.21.15.88:0/3759000778 wait complete. 2024-04-26T20:02:38.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.759+0000 7fda2b200640 1 Processor -- start 2024-04-26T20:02:38.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 -- start start 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda2c15f1b0 0x7fda2c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0d33a0 con 0x7fda2c162b00 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c0b7d90 con 0x7fda2c15f1b0 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.760+0000 7fda2b200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c0d0010 con 0x7fda2c163480 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58278/0 (socket says 172.21.15.88:58278) 2024-04-26T20:02:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 -- 172.21.15.88:0/888190777 learned_addr learned my addr 172.21.15.88:0/888190777 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda2a800640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29e00640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda2c15f1b0 0x7fda2c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 -- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 msgr2=0x7fda2c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 -- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda2c15f1b0 msgr2=0x7fda2c1f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda2c15f1b0 0x7fda2c1f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda29400640 1 -- 172.21.15.88:0/888190777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1f5220 con 0x7fda2c162b00 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.761+0000 7fda2a800640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:02:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda29400640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda1802a220 tx=0x7fda180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18041020 con 0x7fda2c162b00 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda180306c0 con 0x7fda2c162b00 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1f5500 con 0x7fda2c162b00 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18037040 con 0x7fda2c162b00 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.762+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1f5ac0 con 0x7fda2c162b00 2024-04-26T20:02:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.763+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c0693c0 con 0x7fda2c162b00 2024-04-26T20:02:38.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.764+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fda18044020 con 0x7fda2c162b00 2024-04-26T20:02:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.765+0000 7fda22000640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 0x7fda0c06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.765+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fda180abff0 con 0x7fda2c162b00 2024-04-26T20:02:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.766+0000 7fda29e00640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 0x7fda0c06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.766+0000 7fda29e00640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 0x7fda0c06ee90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda1c006fd0 tx=0x7fda1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.769+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda1804b030 con 0x7fda2c162b00 2024-04-26T20:02:38.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:38.967+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fda2c08d090 con 0x7fda2c162b00 2024-04-26T20:02:39.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.127+0000 7fda22000640 1 -- 172.21.15.88:0/888190777 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) ==== 97+0+0 (secure 0 0 0) 0x7fda1807a780 con 0x7fda2c162b00 2024-04-26T20:02:39.128 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.131+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 msgr2=0x7fda0c06ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.131+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 0x7fda0c06ee90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda1c006fd0 tx=0x7fda1c008040 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 msgr2=0x7fda2c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda1802a220 tx=0x7fda180046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 shutdown_connections 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fda0c06c9d0 0x7fda0c06ee90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fda2c163480 0x7fda2c15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda2c162b00 0x7fda2c15d310 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 --2- 172.21.15.88:0/888190777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda2c15f1b0 0x7fda2c1f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 >> 172.21.15.88:0/888190777 conn(0x7fda2c09e230 msgr2=0x7fda2c0d4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 shutdown_connections 2024-04-26T20:02:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.132+0000 7fda2b200640 1 -- 172.21.15.88:0/888190777 wait complete. 2024-04-26T20:02:39.153 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:02:39.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.338+0000 7fe2f7000640 1 Processor -- start 2024-04-26T20:02:39.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.339+0000 7fe2f7000640 1 -- start start 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f7000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2f00cc410 0x7fe2f00c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f7000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f006e680 con 0x7fe2f00ca370 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f7000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f0070130 con 0x7fe2f0147d50 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f7000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f00ca780 con 0x7fe2f00cc410 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f6600640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe2f00ca370 0x7fe2f0168b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50948/0 (socket says 172.21.15.88:50948) 2024-04-26T20:02:39.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.340+0000 7fe2f6600640 1 -- 172.21.15.88:0/3148486946 learned_addr learned my addr 172.21.15.88:0/3148486946 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585543777 0 0) 0x7fe2f006e680 con 0x7fe2f00ca370 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2d8003520 con 0x7fe2f00ca370 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2f5200640 1 --2- 172.21.15.88:0/3148486946 >> v2:172.21.15.132:3300/0 conn(0x7fe2f00cc410 0x7fe2f00c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 116749461 0 0) 0x7fe2d8003520 con 0x7fe2f00ca370 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 >> v2:172.21.15.132:3300/0 conn(0x7fe2f00cc410 msgr2=0x7fe2f00c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 --2- 172.21.15.88:0/3148486946 >> v2:172.21.15.132:3300/0 conn(0x7fe2f00cc410 0x7fe2f00c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 >> v1:172.21.15.132:6789/0 conn(0x7fe2f0147d50 legacy=0x7fe2f00cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f00caa40 con 0x7fe2f00ca370 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fe2e8001a10 con 0x7fe2f00ca370 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 >> v1:172.21.15.88:6789/0 conn(0x7fe2f00ca370 legacy=0x7fe2f0168b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 0x7fe2d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d8004800 0x7fe2d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d80051c0 0x7fe2d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f00ca780 con 0x7fe2d80039b0 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d800c110 con 0x7fe2d80051c0 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.341+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d800c310 con 0x7fe2d8004800 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 0x7fe2d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f6600640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d80051c0 0x7fe2d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5200640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d8004800 0x7fe2d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 -- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d8004800 msgr2=0x7fe2d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d8004800 0x7fe2d8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 -- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d80051c0 msgr2=0x7fe2d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:39.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2d80051c0 0x7fe2d800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f6600640 1 -- 172.21.15.88:0/3148486946 reap_dead start 2024-04-26T20:02:39.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.342+0000 7fe2f5c00640 1 -- 172.21.15.88:0/3148486946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d8003520 con 0x7fe2d80039b0 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.343+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 0x7fe2d8003dd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001750 tx=0x7fe2d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666d2bed79269e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4001370 con 0x7fe2d80039b0 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e40052d0 con 0x7fe2d80039b0 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2efe00640 1 -- 172.21.15.88:0/3148486946 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e402abc0 con 0x7fe2d80039b0 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 -- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 msgr2=0x7fe2d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 0x7fe2d8003dd0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001750 tx=0x7fe2d8005830 comp rx=0 tx=0).stop 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 -- 172.21.15.88:0/3148486946 shutdown_connections 2024-04-26T20:02:39.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 --2- 172.21.15.88:0/3148486946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2d80039b0 0x7fe2d8003dd0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 -- 172.21.15.88:0/3148486946 >> 172.21.15.88:0/3148486946 conn(0x7fe2f00d0ae0 msgr2=0x7fe2f00d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:39.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 -- 172.21.15.88:0/3148486946 shutdown_connections 2024-04-26T20:02:39.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.344+0000 7fe2f7000640 1 -- 172.21.15.88:0/3148486946 wait complete. 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.346+0000 7fe2f7000640 1 Processor -- start 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.346+0000 7fe2f7000640 1 -- start start 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d80039b0 0x7fe2f0134d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f01378d0 0x7fe2f00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006ec00 con 0x7fe2f015c780 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f006e920 con 0x7fe2f01378d0 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f7000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f00ce6b0 con 0x7fe2d80039b0 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60510/0 (socket says 172.21.15.88:60510) 2024-04-26T20:02:39.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.347+0000 7fe2f6600640 1 -- 172.21.15.88:0/4230433108 learned_addr learned my addr 172.21.15.88:0/4230433108 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:39.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 -- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d80039b0 msgr2=0x7fe2f0134d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d80039b0 0x7fe2f0134d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 -- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f01378d0 msgr2=0x7fe2f00b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f01378d0 0x7fe2f00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 -- 172.21.15.88:0/4230433108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f00b4020 con 0x7fe2f015c780 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f6600640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0071e10 tx=0x7fe2e800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800b8d0 con 0x7fe2f015c780 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e800ba70 con 0x7fe2f015c780 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e800be60 con 0x7fe2f015c780 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f00b4300 con 0x7fe2f015c780 2024-04-26T20:02:39.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.348+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f015cc00 con 0x7fe2f015c780 2024-04-26T20:02:39.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.349+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f00b4820 con 0x7fe2f015c780 2024-04-26T20:02:39.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.350+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe2e801b020 con 0x7fe2f015c780 2024-04-26T20:02:39.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.351+0000 7fe2ed600640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 0x7fe2d006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.351+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fe2e808ba20 con 0x7fe2f015c780 2024-04-26T20:02:39.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.351+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 0x7fe2d006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.352+0000 7fe2f5c00640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 0x7fe2d006ee40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4032000 tx=0x7fe2e402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.356+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2f00b4820 con 0x7fe2f015c780 2024-04-26T20:02:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.629+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe2f00693c0 con 0x7fe2f015c780 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.630+0000 7fe2ed600640 1 -- 172.21.15.88:0/4230433108 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fe2e805a130 con 0x7fe2f015c780 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-04-26T20:02:39.631 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0jr1OBAAQRkT9BgmjXURkkrr7MAHWQ==", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmpzxOGBAADbar8X1eXjXWjfB60egiBQ==", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:02:39.632 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmCnKlCxAAo0jGLXT6bRL3qpWufJJ5Sw==", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.633 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmRfxVJhAAGgPI5k+fa3OuMZs2kt0msg==", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-04-26T20:02:39.634 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB3ByxmU+QbKBAAKGuevMSCLF8SYrdlt/62YQ==", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ByxmJeeNMxAA64NhSfVOpwUK5gJ6cQVDvQ==", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.635 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB+Byxm0QbGAhAA6ufvhdQMCNOPOhioCEovNg==", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCBByxmpToFDxAAAuw5Ew77CrBtkai6YP0AlA==", 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.636 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCHByxmou+EHBAAkwkBnJbDkuczBKTj/szbYA==", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.637 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCKByxmNUjQNxAAz8gNq0pCsDTYh5Fo7xzwbg==", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCOByxmDaIOFxAAoLuo0jYUxlFBf/2VpwDkPw==", 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.638 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCRByxmaXFiIBAAb4ALfWrAFga124RrQFb4Jw==", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.639 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw==", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0ByxmbJEcCRAA2X7/8G8AR+PRlmM/59Ebqg==", 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.640 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBzByxmEZsPOxAAJZIw3+ol9NxsWyB851hSRg==", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:02:39.641 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB0Byxm+tkaEhAAXoURy4RGA/OZsj7sUhk8ZQ==", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-04-26T20:02:39.642 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQBxByxmqnYUOBAAkUPOrzy4lbEq83+B/FNZlw==", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0kFbLxAApmvQFSMZqcrF7d0MGYbgQw==", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:02:39.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQByByxm0PeZJhAAmWcbnXTpRhnfZRdvKzbTTg==", 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:39.644 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-26T20:02:39.645 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 msgr2=0x7fe2d006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 0x7fe2d006ee40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4032000 tx=0x7fe2e402c000 comp rx=0 tx=0).stop 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 msgr2=0x7fe2f015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0071e10 tx=0x7fe2e800b0d0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 shutdown_connections 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2f015c780 0x7fe2f015d5d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2f01378d0 0x7fe2f00b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2d006c980 0x7fe2d006ee40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 --2- 172.21.15.88:0/4230433108 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2d80039b0 0x7fe2f0134d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 >> 172.21.15.88:0/4230433108 conn(0x7fe2f00d0ae0 msgr2=0x7fe2f015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:39.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 shutdown_connections 2024-04-26T20:02:39.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.633+0000 7fe2f7000640 1 -- 172.21.15.88:0/4230433108 wait complete. 2024-04-26T20:02:39.651 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:39.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.856+0000 7fdb89600640 1 Processor -- start 2024-04-26T20:02:39.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.856+0000 7fdb89600640 1 -- start start 2024-04-26T20:02:39.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.857+0000 7fdb89600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdb8406c770 0x7fdb8406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.857+0000 7fdb89600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cfd0 0x7fdb84073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb89600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb8406d090 con 0x7fdb8415cb70 2024-04-26T20:02:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb89600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fdb840af460 con 0x7fdb8406c770 2024-04-26T20:02:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb89600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdb840ae850 con 0x7fdb8415cfd0 2024-04-26T20:02:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fdb8406c770 0x7fdb8406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb7a000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fdb8415cb70 0x7fdb84073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50956/0 (socket says 172.21.15.88:50956) 2024-04-26T20:02:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb7a000640 1 -- 172.21.15.88:0/3116533998 learned_addr learned my addr 172.21.15.88:0/3116533998 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb83400640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cfd0 0x7fdb84073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82000640 1 -- 172.21.15.88:0/3116533998 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1871617008 0 0) 0x7fdb8406d090 con 0x7fdb8415cb70 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82000640 1 -- 172.21.15.88:0/3116533998 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb64003520 con 0x7fdb8415cb70 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82a00640 1 -- 172.21.15.88:0/3116533998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cfd0 msgr2=0x7fdb84073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82a00640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cfd0 0x7fdb84073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82a00640 1 -- 172.21.15.88:0/3116533998 >> v1:172.21.15.88:6789/0 conn(0x7fdb8415cb70 legacy=0x7fdb84073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.858+0000 7fdb82a00640 1 -- 172.21.15.88:0/3116533998 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840aeb30 con 0x7fdb8406c770 2024-04-26T20:02:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb82a00640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8406c770 0x7fdb8406cb50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb740097a0 tx=0x7fdb7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff687c5a4bed98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb82000640 1 -- 172.21.15.88:0/3116533998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74037070 con 0x7fdb8406c770 2024-04-26T20:02:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb82000640 1 -- 172.21.15.88:0/3116533998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb74032440 con 0x7fdb8406c770 2024-04-26T20:02:39.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb82000640 1 -- 172.21.15.88:0/3116533998 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74032810 con 0x7fdb8406c770 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 -- 172.21.15.88:0/3116533998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8406c770 msgr2=0x7fdb8406cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8406c770 0x7fdb8406cb50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fdb740097a0 tx=0x7fdb7402a180 comp rx=0 tx=0).stop 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 -- 172.21.15.88:0/3116533998 shutdown_connections 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cfd0 0x7fdb84073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 --2- 172.21.15.88:0/3116533998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8406c770 0x7fdb8406cb50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.860+0000 7fdb89600640 1 -- 172.21.15.88:0/3116533998 >> 172.21.15.88:0/3116533998 conn(0x7fdb840b2d50 msgr2=0x7fdb840b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.861+0000 7fdb89600640 1 -- 172.21.15.88:0/3116533998 shutdown_connections 2024-04-26T20:02:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.861+0000 7fdb89600640 1 -- 172.21.15.88:0/3116533998 wait complete. 2024-04-26T20:02:39.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.862+0000 7fdb89600640 1 Processor -- start 2024-04-26T20:02:39.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.862+0000 7fdb89600640 1 -- start start 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 0x7fdb8414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cb70 0x7fdb840cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8415c600 con 0x7fdb8415cfd0 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840af0f0 con 0x7fdb8406c770 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.863+0000 7fdb89600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8415c780 con 0x7fdb8415cb70 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60524/0 (socket says 172.21.15.88:60524) 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 -- 172.21.15.88:0/4061508169 learned_addr learned my addr 172.21.15.88:0/4061508169 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb82a00640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 0x7fdb8414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb7a000640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cb70 0x7fdb840cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 -- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cb70 msgr2=0x7fdb840cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cb70 0x7fdb840cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 -- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 msgr2=0x7fdb8414ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 0x7fdb8414ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb83400640 1 -- 172.21.15.88:0/4061508169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840cead0 con 0x7fdb8415cfd0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.864+0000 7fdb82a00640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 0x7fdb8414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb83400640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c007e60 tx=0x7fdb7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c014420 con 0x7fdb8415cfd0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c019070 con 0x7fdb8415cfd0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c014820 con 0x7fdb8415cfd0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb840cda30 con 0x7fdb8415cfd0 2024-04-26T20:02:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.865+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb840cdff0 con 0x7fdb8415cfd0 2024-04-26T20:02:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.866+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84069540 con 0x7fdb8415cfd0 2024-04-26T20:02:39.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.867+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fdb7c0149c0 con 0x7fdb8415cfd0 2024-04-26T20:02:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.868+0000 7fdb7b400640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 0x7fdb6006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.868+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fdb7c08c180 con 0x7fdb8415cfd0 2024-04-26T20:02:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.869+0000 7fdb82a00640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 0x7fdb6006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.869+0000 7fdb82a00640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 0x7fdb6006ee70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb74034000 tx=0x7fdb7402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:39.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:39.873+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb7c05a810 con 0x7fdb8415cfd0 2024-04-26T20:02:40.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:02:40.055 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:02:40.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.075+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb841605c0 con 0x7fdb8415cfd0 2024-04-26T20:02:40.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.076+0000 7fdb7b400640 1 -- 172.21.15.88:0/4061508169 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12879 (secure 0 0 0) 0x7fdb7c05a110 con 0x7fdb8415cfd0 2024-04-26T20:02:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:40.077 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":59,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:39.140169+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:40.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.078+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 msgr2=0x7fdb6006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.078+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 0x7fdb6006ee70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdb74034000 tx=0x7fdb7402e000 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.078+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 msgr2=0x7fdb840d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.078+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c007e60 tx=0x7fdb7c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 shutdown_connections 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8415cfd0 0x7fdb840d0bb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fdb6006c9b0 0x7fdb6006ee70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fdb8415cb70 0x7fdb840cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 --2- 172.21.15.88:0/4061508169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb8406c770 0x7fdb8414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 >> 172.21.15.88:0/4061508169 conn(0x7fdb840b2d50 msgr2=0x7fdb8415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 shutdown_connections 2024-04-26T20:02:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.079+0000 7fdb89600640 1 -- 172.21.15.88:0/4061508169 wait complete. 2024-04-26T20:02:40.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:02:40.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:02:40.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.330+0000 7fcccf200640 1 Processor -- start 2024-04-26T20:02:40.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.330+0000 7fcccf200640 1 -- start start 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccf200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fccd0161140 0x7fccd006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccf200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fccd0146280 0x7fccd01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccf200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd00b4d20 con 0x7fccd0160d60 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccf200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fccd0146c30 con 0x7fccd0146280 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccf200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fccd00b5000 con 0x7fccd0161140 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccde00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fccd0160d60 0x7fccd00b5b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50968/0 (socket says 172.21.15.88:50968) 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fccce800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fccd0146280 0x7fccd01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fcccde00640 1 -- 172.21.15.88:0/1667192618 learned_addr learned my addr 172.21.15.88:0/1667192618 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:40.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.332+0000 7fccc5400640 1 --2- 172.21.15.88:0/1667192618 >> v2:172.21.15.132:3300/0 conn(0x7fccd0161140 0x7fccd006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.333+0000 7fccce800640 1 -- 172.21.15.88:0/1667192618 >> v2:172.21.15.132:3300/0 conn(0x7fccd0161140 msgr2=0x7fccd006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:40.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.333+0000 7fccce800640 1 --2- 172.21.15.88:0/1667192618 >> v2:172.21.15.132:3300/0 conn(0x7fccd0161140 0x7fccd006dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.333+0000 7fccce800640 1 -- 172.21.15.88:0/1667192618 >> v1:172.21.15.88:6789/0 conn(0x7fccd0160d60 legacy=0x7fccd00b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.333+0000 7fccce800640 1 -- 172.21.15.88:0/1667192618 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd00b52e0 con 0x7fccd0146280 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.333+0000 7fcccd400640 1 -- 172.21.15.88:0/1667192618 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942448194 0 0) 0x7fccd00b4d20 con 0x7fccd0160d60 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fccce800640 1 --2- 172.21.15.88:0/1667192618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0146280 0x7fccd01466f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009d90 tx=0x7fccc802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbad20bb17ceda86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fcccd400640 1 -- 172.21.15.88:0/1667192618 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8007740 con 0x7fccd0146280 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fcccd400640 1 -- 172.21.15.88:0/1667192618 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc80078e0 con 0x7fccd0146280 2024-04-26T20:02:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fcccd400640 1 -- 172.21.15.88:0/1667192618 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8007cb0 con 0x7fccd0146280 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fcccf200640 1 -- 172.21.15.88:0/1667192618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0146280 msgr2=0x7fccd01466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.334+0000 7fcccf200640 1 --2- 172.21.15.88:0/1667192618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0146280 0x7fccd01466f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009d90 tx=0x7fccc802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 -- 172.21.15.88:0/1667192618 shutdown_connections 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 --2- 172.21.15.88:0/1667192618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0146280 0x7fccd01466f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 --2- 172.21.15.88:0/1667192618 >> v2:172.21.15.132:3300/0 conn(0x7fccd0161140 0x7fccd006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 -- 172.21.15.88:0/1667192618 >> 172.21.15.88:0/1667192618 conn(0x7fccd00b2900 msgr2=0x7fccd00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 -- 172.21.15.88:0/1667192618 shutdown_connections 2024-04-26T20:02:40.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.335+0000 7fcccf200640 1 -- 172.21.15.88:0/1667192618 wait complete. 2024-04-26T20:02:40.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.336+0000 7fcccf200640 1 Processor -- start 2024-04-26T20:02:40.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 -- start start 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccd0146280 0x7fccd0135130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccd0160d60 0x7fccd0071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fccd006e420 con 0x7fccd0161140 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fccd00cf6f0 con 0x7fccd0160d60 2024-04-26T20:02:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.337+0000 7fcccf200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fccd006e5a0 con 0x7fccd0146280 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60536/0 (socket says 172.21.15.88:60536) 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 -- 172.21.15.88:0/2138536583 learned_addr learned my addr 172.21.15.88:0/2138536583 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fcccde00640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccd0146280 0x7fccd0135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccc5400640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccd0160d60 0x7fccd0071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 -- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccd0146280 msgr2=0x7fccd0135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccd0146280 0x7fccd0135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 -- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccd0160d60 msgr2=0x7fccd0071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccd0160d60 0x7fccd0071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.338+0000 7fccce800640 1 -- 172.21.15.88:0/2138536583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd00cd240 con 0x7fccd0161140 2024-04-26T20:02:40.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fccce800640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009eb0 tx=0x7fccc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8040070 con 0x7fccd0161140 2024-04-26T20:02:40.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc8031bc0 con 0x7fccd0161140 2024-04-26T20:02:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8009450 con 0x7fccd0161140 2024-04-26T20:02:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd00cad10 con 0x7fccd0161140 2024-04-26T20:02:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.339+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd00cd7a0 con 0x7fccd0161140 2024-04-26T20:02:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.340+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd00693c0 con 0x7fccd0161140 2024-04-26T20:02:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.341+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fccc8039040 con 0x7fccd0161140 2024-04-26T20:02:40.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.342+0000 7fccc6a00640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 0x7fccac06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.342+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fccc80ad110 con 0x7fccd0161140 2024-04-26T20:02:40.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.342+0000 7fcccde00640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 0x7fccac06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.343+0000 7fcccde00640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 0x7fccac06ee40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000f60 tx=0x7fccc0007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.346+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccd00cd240 con 0x7fccd0161140 2024-04-26T20:02:40.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.547+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fccd00cfbb0 con 0x7fccd0161140 2024-04-26T20:02:40.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.547+0000 7fccc6a00640 1 -- 172.21.15.88:0/2138536583 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fccc807b820 con 0x7fccd0161140 2024-04-26T20:02:40.548 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 msgr2=0x7fccac06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 0x7fccac06ee40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000f60 tx=0x7fccc0007b50 comp rx=0 tx=0).stop 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 msgr2=0x7fccd00723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009eb0 tx=0x7fccc8004a40 comp rx=0 tx=0).stop 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 shutdown_connections 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccd0161140 0x7fccd00723c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fccd0160d60 0x7fccd0071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fccac06c980 0x7fccac06ee40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 --2- 172.21.15.88:0/2138536583 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fccd0146280 0x7fccd0135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 >> 172.21.15.88:0/2138536583 conn(0x7fccd00b2900 msgr2=0x7fccd015fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:40.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 shutdown_connections 2024-04-26T20:02:40.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.550+0000 7fcccf200640 1 -- 172.21.15.88:0/2138536583 wait complete. 2024-04-26T20:02:40.570 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:02:40.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.766+0000 7f7f43800640 1 Processor -- start 2024-04-26T20:02:40.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.766+0000 7f7f43800640 1 -- start start 2024-04-26T20:02:40.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.767+0000 7f7f43800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c15dc80 0x7f7f3c167c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.767+0000 7f7f43800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7f3c164240 0x7f7f3c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.767+0000 7f7f43800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c164740 con 0x7f7f3c06c770 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f43800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3c0b5ff0 con 0x7f7f3c164240 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f43800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c06d110 con 0x7f7f3c15dc80 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7f3c164240 0x7f7f3c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7f3c164240 0x7f7f3c06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60546/0 (socket says 172.21.15.88:60546) 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 -- 172.21.15.88:0/1287680453 learned_addr learned my addr 172.21.15.88:0/1287680453 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f40e00640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c15dc80 0x7f7f3c167c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 -- 172.21.15.88:0/1287680453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c15dc80 msgr2=0x7f7f3c167c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c15dc80 0x7f7f3c167c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.768+0000 7f7f41800640 1 -- 172.21.15.88:0/1287680453 >> v1:172.21.15.132:6789/0 conn(0x7f7f3c06c770 legacy=0x7f7f3c163b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.769+0000 7f7f41800640 1 -- 172.21.15.88:0/1287680453 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c1671a0 con 0x7f7f3c164240 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f41800640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c164240 0x7f7f3c06cbd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009870 tx=0x7f7f2c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b505ad6a3b6f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f3b400640 1 -- 172.21.15.88:0/1287680453 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c037070 con 0x7f7f3c164240 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f3b400640 1 -- 172.21.15.88:0/1287680453 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2c032440 con 0x7f7f3c164240 2024-04-26T20:02:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f3b400640 1 -- 172.21.15.88:0/1287680453 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c032810 con 0x7f7f3c164240 2024-04-26T20:02:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f43800640 1 -- 172.21.15.88:0/1287680453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c164240 msgr2=0x7f7f3c06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.770+0000 7f7f43800640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c164240 0x7f7f3c06cbd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009870 tx=0x7f7f2c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 -- 172.21.15.88:0/1287680453 shutdown_connections 2024-04-26T20:02:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c164240 0x7f7f3c06cbd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 --2- 172.21.15.88:0/1287680453 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c15dc80 0x7f7f3c167c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 -- 172.21.15.88:0/1287680453 >> 172.21.15.88:0/1287680453 conn(0x7f7f3c0b2d50 msgr2=0x7f7f3c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 -- 172.21.15.88:0/1287680453 shutdown_connections 2024-04-26T20:02:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.771+0000 7f7f43800640 1 -- 172.21.15.88:0/1287680453 wait complete. 2024-04-26T20:02:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.772+0000 7f7f43800640 1 Processor -- start 2024-04-26T20:02:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.773+0000 7f7f43800640 1 -- start start 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c15dc80 0x7f7f3c14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 0x7f7f3c14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c164a20 con 0x7f7f3c06c770 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c167760 con 0x7f7f3c15dc80 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f43800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c0add80 con 0x7f7f3c164240 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60560/0 (socket says 172.21.15.88:60560) 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 -- 172.21.15.88:0/561895115 learned_addr learned my addr 172.21.15.88:0/561895115 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f3be00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c15dc80 0x7f7f3c14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f41800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 0x7f7f3c14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 -- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 msgr2=0x7f7f3c14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 0x7f7f3c14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 -- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c15dc80 msgr2=0x7f7f3c14ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c15dc80 0x7f7f3c14ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f40e00640 1 -- 172.21.15.88:0/561895115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c0ce990 con 0x7f7f3c06c770 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.774+0000 7f7f41800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 0x7f7f3c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f40e00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d800 tx=0x7f7f3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400b6c0 con 0x7f7f3c06c770 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34019070 con 0x7f7f3c06c770 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c0cec70 con 0x7f7f3c06c770 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400ba80 con 0x7f7f3c06c770 2024-04-26T20:02:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.775+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c0cf230 con 0x7f7f3c06c770 2024-04-26T20:02:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.776+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c069540 con 0x7f7f3c06c770 2024-04-26T20:02:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.777+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7f3400bc20 con 0x7f7f3c06c770 2024-04-26T20:02:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.778+0000 7f7f38c00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 0x7f7f1806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.778+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f7f3408bec0 con 0x7f7f3c06c770 2024-04-26T20:02:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.779+0000 7f7f3be00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 0x7f7f1806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.779+0000 7f7f3be00640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 0x7f7f1806ee40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0d0080 tx=0x7f7f24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:40.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.783+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f3405a4c0 con 0x7f7f3c06c770 2024-04-26T20:02:40.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.985+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7f3c0b32d0 con 0x7f7f3c06c770 2024-04-26T20:02:40.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.986+0000 7f7f38c00640 1 -- 172.21.15.88:0/561895115 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) ==== 84+0+65 (secure 0 0 0) 0x7f7f34059dc0 con 0x7f7f3c06c770 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:40.987 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:02:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 msgr2=0x7f7f1806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 0x7f7f1806ee40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0d0080 tx=0x7f7f24009040 comp rx=0 tx=0).stop 2024-04-26T20:02:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 msgr2=0x7f7f3c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d800 tx=0x7f7f3400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 shutdown_connections 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7f1806c980 0x7f7f1806ee40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7f3c164240 0x7f7f3c14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c15dc80 0x7f7f3c14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 --2- 172.21.15.88:0/561895115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c06c770 0x7f7f3c0cfa30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.988+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 >> 172.21.15.88:0/561895115 conn(0x7f7f3c0b2d50 msgr2=0x7f7f3c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.989+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 shutdown_connections 2024-04-26T20:02:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:40.989+0000 7f7f43800640 1 -- 172.21.15.88:0/561895115 wait complete. 2024-04-26T20:02:41.010 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:02:41.010 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:02:41.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.215+0000 7f7e09200640 1 Processor -- start 2024-04-26T20:02:41.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.216+0000 7f7e09200640 1 -- start start 2024-04-26T20:02:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.217+0000 7f7e09200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7e04142600 0x7f7e040cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.217+0000 7f7e09200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f7e040cb6e0 0x7f7e0406f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.217+0000 7f7e09200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e040cbc60 con 0x7f7e0406fda0 2024-04-26T20:02:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.217+0000 7f7e09200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f7e04070210 con 0x7f7e04142600 2024-04-26T20:02:41.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.217+0000 7f7e09200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f7e040ccd00 con 0x7f7e040cb6e0 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7e04142600 0x7f7e040cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f7e04142600 0x7f7e040cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60574/0 (socket says 172.21.15.88:60574) 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 -- 172.21.15.88:0/1843824492 learned_addr learned my addr 172.21.15.88:0/1843824492 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 -- 172.21.15.88:0/1843824492 >> v2:172.21.15.132:3300/0 conn(0x7f7e040cb6e0 msgr2=0x7f7e0406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 --2- 172.21.15.88:0/1843824492 >> v2:172.21.15.132:3300/0 conn(0x7f7e040cb6e0 0x7f7e0406f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 -- 172.21.15.88:0/1843824492 >> v1:172.21.15.132:6789/0 conn(0x7f7e0406fda0 legacy=0x7f7e0415fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:41.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.218+0000 7f7e03400640 1 -- 172.21.15.88:0/1843824492 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e040cbf40 con 0x7f7e04142600 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.219+0000 7f7e03400640 1 --2- 172.21.15.88:0/1843824492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e04142600 0x7f7e040cb1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406a3b0 tx=0x7f7df402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b816e1c2edb65e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.219+0000 7f7e02a00640 1 -- 172.21.15.88:0/1843824492 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4032710 con 0x7f7e04142600 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e02a00640 1 -- 172.21.15.88:0/1843824492 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df40328b0 con 0x7f7e04142600 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e02a00640 1 -- 172.21.15.88:0/1843824492 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4032ca0 con 0x7f7e04142600 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 -- 172.21.15.88:0/1843824492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e04142600 msgr2=0x7f7e040cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 --2- 172.21.15.88:0/1843824492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e04142600 0x7f7e040cb1a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7e0406a3b0 tx=0x7f7df402c850 comp rx=0 tx=0).stop 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 -- 172.21.15.88:0/1843824492 shutdown_connections 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 --2- 172.21.15.88:0/1843824492 >> v2:172.21.15.132:3300/0 conn(0x7f7e040cb6e0 0x7f7e0406f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 --2- 172.21.15.88:0/1843824492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e04142600 0x7f7e040cb1a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 -- 172.21.15.88:0/1843824492 >> 172.21.15.88:0/1843824492 conn(0x7f7e040d0b50 msgr2=0x7f7e040d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 -- 172.21.15.88:0/1843824492 shutdown_connections 2024-04-26T20:02:41.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.220+0000 7f7e09200640 1 -- 172.21.15.88:0/1843824492 wait complete. 2024-04-26T20:02:41.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.222+0000 7f7e09200640 1 Processor -- start 2024-04-26T20:02:41.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.222+0000 7f7e09200640 1 -- start start 2024-04-26T20:02:41.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e040cb6e0 0x7f7e041411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e04142600 0x7f7e04141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7e040cc7e0 con 0x7f7e0406fda0 2024-04-26T20:02:41.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f7e040cc500 con 0x7f7e040cb6e0 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e09200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f7e04072de0 con 0x7f7e04142600 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e03400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e03400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60588/0 (socket says 172.21.15.88:60588) 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.223+0000 7f7e03400640 1 -- 172.21.15.88:0/3849794464 learned_addr learned my addr 172.21.15.88:0/3849794464 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 -- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e04142600 msgr2=0x7f7e04141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e04142600 0x7f7e04141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 -- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e040cb6e0 msgr2=0x7f7e041411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e040cb6e0 0x7f7e041411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:41.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 -- 172.21.15.88:0/3849794464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e040c5d70 con 0x7f7e0406fda0 2024-04-26T20:02:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e03400640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7df402c890 tx=0x7f7df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4041070 con 0x7f7e0406fda0 2024-04-26T20:02:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df4003810 con 0x7f7e0406fda0 2024-04-26T20:02:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4003c00 con 0x7f7e0406fda0 2024-04-26T20:02:41.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.224+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e041ea290 con 0x7f7e0406fda0 2024-04-26T20:02:41.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.225+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e041eab30 con 0x7f7e0406fda0 2024-04-26T20:02:41.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.226+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e041ea680 con 0x7f7e0406fda0 2024-04-26T20:02:41.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.227+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7df4044070 con 0x7f7e0406fda0 2024-04-26T20:02:41.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.228+0000 7f7dfb400640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 0x7f7de006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:41.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.228+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f7df40ac940 con 0x7f7e0406fda0 2024-04-26T20:02:41.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.228+0000 7f7dfbe00640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 0x7f7de006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:41.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.229+0000 7f7dfbe00640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 0x7f7de006ee70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e040b61b0 tx=0x7f7dec006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:41.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.232+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7df407afe0 con 0x7f7e0406fda0 2024-04-26T20:02:41.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:41.432+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7e0406d1f0 con 0x7f7e0406fda0 2024-04-26T20:02:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.153+0000 7f7dfb400640 1 -- 172.21.15.88:0/3849794464 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) ==== 138+0+0 (secure 0 0 0) 0x7f7e0406d1f0 con 0x7f7e0406fda0 2024-04-26T20:02:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-04-26T20:02:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 msgr2=0x7f7de006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 0x7f7de006ee70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e040b61b0 tx=0x7f7dec006bf0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 msgr2=0x7f7e041e7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7df402c890 tx=0x7f7df40046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 shutdown_connections 2024-04-26T20:02:42.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f7de006c9b0 0x7f7de006ee70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f7e04142600 0x7f7e04141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e040cb6e0 0x7f7e041411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 --2- 172.21.15.88:0/3849794464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e0406fda0 0x7f7e041e7eb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.157+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 >> 172.21.15.88:0/3849794464 conn(0x7f7e040d0b50 msgr2=0x7f7e04158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.158+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 shutdown_connections 2024-04-26T20:02:42.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.158+0000 7f7e09200640 1 -- 172.21.15.88:0/3849794464 wait complete. 2024-04-26T20:02:42.184 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:02:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.377+0000 7fa5e1a00640 1 Processor -- start 2024-04-26T20:02:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.377+0000 7fa5e1a00640 1 -- start start 2024-04-26T20:02:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5dc0d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0d15d0 0x7fa5dc0d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc0d0a10 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc0d0cf0 con 0x7fa5dc0d15d0 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b4a80 con 0x7fa5dc162850 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa5dc0d2c80 0x7fa5dc0b67d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50972/0 (socket says 172.21.15.88:50972) 2024-04-26T20:02:42.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5e1000640 1 -- 172.21.15.88:0/3887132116 learned_addr learned my addr 172.21.15.88:0/3887132116 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5dc0d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.379+0000 7fa5db400640 1 --2- 172.21.15.88:0/3887132116 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0d15d0 0x7fa5dc0d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2261526785 0 0) 0x7fa5dc0d0a10 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5c0003520 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777971247 0 0) 0x7fa5c0003520 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 msgr2=0x7fa5dc0d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5dc0d3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0d15d0 msgr2=0x7fa5dc0d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 --2- 172.21.15.88:0/3887132116 >> v2:172.21.15.132:3300/0 conn(0x7fa5dc0d15d0 0x7fa5dc0d39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc0b4d60 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa5d40019f0 con 0x7fa5dc0d2c80 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 >> v1:172.21.15.88:6789/0 conn(0x7fa5dc0d2c80 legacy=0x7fa5dc0b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c00039b0 0x7fa5c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0d0cf0 con 0x7fa5c0004800 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b4a80 con 0x7fa5c00039b0 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.380+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c000ba70 con 0x7fa5dc162850 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c00039b0 0x7fa5c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5e1000640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 -- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 msgr2=0x7fa5c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5c00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 -- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c00039b0 msgr2=0x7fa5c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c00039b0 0x7fa5c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5db400640 1 -- 172.21.15.88:0/3887132116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c0003520 con 0x7fa5c0004800 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5c00039b0 0x7fa5c0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.381+0000 7fa5e1000640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc162850 0x7fa5c00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.382+0000 7fa5db400640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009a90 tx=0x7fa5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469617353549c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.382+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8036070 con 0x7fa5c0004800 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.382+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c8031400 con 0x7fa5c0004800 2024-04-26T20:02:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.382+0000 7fa5daa00640 1 -- 172.21.15.88:0/3887132116 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8009690 con 0x7fa5c0004800 2024-04-26T20:02:42.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 -- 172.21.15.88:0/3887132116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 msgr2=0x7fa5c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/3887132116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5c0004c80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009a90 tx=0x7fa5c0005830 comp rx=0 tx=0).stop 2024-04-26T20:02:42.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1000640 1 -- 172.21.15.88:0/3887132116 reap_dead start 2024-04-26T20:02:42.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 -- 172.21.15.88:0/3887132116 shutdown_connections 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 -- 172.21.15.88:0/3887132116 >> 172.21.15.88:0/3887132116 conn(0x7fa5dc0d4f30 msgr2=0x7fa5dc0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 -- 172.21.15.88:0/3887132116 shutdown_connections 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.383+0000 7fa5e1a00640 1 -- 172.21.15.88:0/3887132116 wait complete. 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.385+0000 7fa5e1a00640 1 Processor -- start 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.385+0000 7fa5e1a00640 1 -- start start 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 0x7fa5dc1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 0x7fa5dc0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0d0740 con 0x7fa5c0004800 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0d05c0 con 0x7fa5dc15d460 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5e1a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b6290 con 0x7fa5dc0b8d50 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60614/0 (socket says 172.21.15.88:60614) 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.386+0000 7fa5dbe00640 1 -- 172.21.15.88:0/1793484121 learned_addr learned my addr 172.21.15.88:0/1793484121 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5e1000640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 0x7fa5dc0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5db400640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 0x7fa5dc1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 -- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 msgr2=0x7fa5dc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 0x7fa5dc0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 -- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 msgr2=0x7fa5dc1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 0x7fa5dc1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 -- 172.21.15.88:0/1793484121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc154900 con 0x7fa5c0004800 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5db400640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 0x7fa5dc1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5e1000640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 0x7fa5dc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5dbe00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00e7e0 tx=0x7fa5cc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.387+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc017070 con 0x7fa5c0004800 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.388+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc00c6d0 con 0x7fa5c0004800 2024-04-26T20:02:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.388+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00f040 con 0x7fa5c0004800 2024-04-26T20:02:42.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.388+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc0b7cb0 con 0x7fa5c0004800 2024-04-26T20:02:42.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.388+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc0b8270 con 0x7fa5c0004800 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.389+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc050760 con 0x7fa5c0004800 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.391+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa5cc01a070 con 0x7fa5c0004800 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.392+0000 7fa5d3e00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 0x7fa5bc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.392+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5227+0+0 (secure 0 0 0) 0x7fa5cc08b600 con 0x7fa5c0004800 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.392+0000 7fa5db400640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 0x7fa5bc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:42.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.393+0000 7fa5db400640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 0x7fa5bc06ee70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8033000 tx=0x7fa5c8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:42.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.397+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5dc154900 con 0x7fa5c0004800 2024-04-26T20:02:42.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:42.600+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa5dc0d3260 con 0x7fa5c0004800 2024-04-26T20:02:43.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.160+0000 7fa5d3e00640 1 -- 172.21.15.88:0/1793484121 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) ==== 170+0+0 (secure 0 0 0) 0x7fa5cc059b10 con 0x7fa5c0004800 2024-04-26T20:02:43.161 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.164+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 msgr2=0x7fa5bc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.164+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 0x7fa5bc06ee70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8033000 tx=0x7fa5c8008030 comp rx=0 tx=0).stop 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.164+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 msgr2=0x7fa5dc0ba9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.164+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00e7e0 tx=0x7fa5cc00eca0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 shutdown_connections 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa5bc06c9b0 0x7fa5bc06ee70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa5dc0b8d50 0x7fa5dc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5dc15d460 0x7fa5dc1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 --2- 172.21.15.88:0/1793484121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5c0004800 0x7fa5dc0ba9a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.165+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 >> 172.21.15.88:0/1793484121 conn(0x7fa5dc0d4f30 msgr2=0x7fa5dc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:43.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.168+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 shutdown_connections 2024-04-26T20:02:43.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.168+0000 7fa5e1a00640 1 -- 172.21.15.88:0/1793484121 wait complete. 2024-04-26T20:02:43.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:02:43.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.386+0000 7f38f4200640 1 Processor -- start 2024-04-26T20:02:43.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.386+0000 7f38f4200640 1 -- start start 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.387+0000 7f38f4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec156650 0x7f38ec15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.387+0000 7f38f4200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f38ec06c770 0x7f38ec06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.387+0000 7f38f4200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38ec06eea0 con 0x7f38ec073120 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.387+0000 7f38f4200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f38ec0b4c40 con 0x7f38ec06c770 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.387+0000 7f38f4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec06f140 con 0x7f38ec156650 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f38ec06c770 0x7f38ec06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f38ec06c770 0x7f38ec06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60624/0 (socket says 172.21.15.88:60624) 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 -- 172.21.15.88:0/3111030146 learned_addr learned my addr 172.21.15.88:0/3111030146 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f1800640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec156650 0x7f38ec15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 -- 172.21.15.88:0/3111030146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec156650 msgr2=0x7f38ec15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec156650 0x7f38ec15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 -- 172.21.15.88:0/3111030146 >> v1:172.21.15.132:6789/0 conn(0x7f38ec073120 legacy=0x7f38ec0735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.388+0000 7f38f0e00640 1 -- 172.21.15.88:0/3111030146 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec06f420 con 0x7f38ec06c770 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38f0e00640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec06c770 0x7f38ec06cbf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009870 tx=0x7f38d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac6b9614e83596 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38ebe00640 1 -- 172.21.15.88:0/3111030146 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8037070 con 0x7f38ec06c770 2024-04-26T20:02:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38ebe00640 1 -- 172.21.15.88:0/3111030146 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d8032440 con 0x7f38ec06c770 2024-04-26T20:02:43.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38ebe00640 1 -- 172.21.15.88:0/3111030146 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8032810 con 0x7f38ec06c770 2024-04-26T20:02:43.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38f4200640 1 -- 172.21.15.88:0/3111030146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec06c770 msgr2=0x7f38ec06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.390+0000 7f38f4200640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec06c770 0x7f38ec06cbf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009870 tx=0x7f38d802a270 comp rx=0 tx=0).stop 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 -- 172.21.15.88:0/3111030146 shutdown_connections 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec06c770 0x7f38ec06cbf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 --2- 172.21.15.88:0/3111030146 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec156650 0x7f38ec15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 -- 172.21.15.88:0/3111030146 >> 172.21.15.88:0/3111030146 conn(0x7f38ec0b2d50 msgr2=0x7f38ec0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 -- 172.21.15.88:0/3111030146 shutdown_connections 2024-04-26T20:02:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.391+0000 7f38f4200640 1 -- 172.21.15.88:0/3111030146 wait complete. 2024-04-26T20:02:43.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.392+0000 7f38f4200640 1 Processor -- start 2024-04-26T20:02:43.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.393+0000 7f38f4200640 1 -- start start 2024-04-26T20:02:43.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.393+0000 7f38f4200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec06c770 0x7f38ec155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.393+0000 7f38f4200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ec073120 0x7f38ec15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f4200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f4200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec06fcc0 con 0x7f38ec156650 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f4200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec06f9e0 con 0x7f38ec073120 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f4200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec15d320 con 0x7f38ec06c770 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60636/0 (socket says 172.21.15.88:60636) 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 -- 172.21.15.88:0/1202537802 learned_addr learned my addr 172.21.15.88:0/1202537802 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f0e00640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ec073120 0x7f38ec15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f1800640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec06c770 0x7f38ec155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 -- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec06c770 msgr2=0x7f38ec155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec06c770 0x7f38ec155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 -- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ec073120 msgr2=0x7f38ec15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ec073120 0x7f38ec15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.394+0000 7f38f2200640 1 -- 172.21.15.88:0/1202537802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec145280 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38f2200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c7d0 tx=0x7f38e400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4003b70 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e4003d30 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec15b250 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400aa90 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.395+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec15b810 con 0x7f38ec156650 2024-04-26T20:02:43.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.396+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec069650 con 0x7f38ec156650 2024-04-26T20:02:43.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.397+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f38e400ac30 con 0x7f38ec156650 2024-04-26T20:02:43.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.398+0000 7f38e9600640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 0x7f38cc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:43.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.398+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5609+0+0 (secure 0 0 0) 0x7f38e408bb20 con 0x7f38ec156650 2024-04-26T20:02:43.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.399+0000 7f38f1800640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 0x7f38cc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:43.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.399+0000 7f38f1800640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 0x7f38cc06ee40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f38e0006fd0 tx=0x7f38e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:43.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.403+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38e4059f50 con 0x7f38ec156650 2024-04-26T20:02:43.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:43.646+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f38ec15cbc0 con 0x7f38ec156650 2024-04-26T20:02:44.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.200+0000 7f38e9600640 1 -- 172.21.15.88:0/1202537802 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v35) ==== 174+0+0 (secure 0 0 0) 0x7f38e4059850 con 0x7f38ec156650 2024-04-26T20:02:44.202 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 10 and data pool 11 2024-04-26T20:02:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.205+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 msgr2=0x7f38cc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.205+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 0x7f38cc06ee40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f38e0006fd0 tx=0x7f38e0008040 comp rx=0 tx=0).stop 2024-04-26T20:02:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.205+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 msgr2=0x7f38ec152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.205+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c7d0 tx=0x7f38e400cc90 comp rx=0 tx=0).stop 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 shutdown_connections 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38ec156650 0x7f38ec152d10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38ec073120 0x7f38ec15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f38cc06c980 0x7f38cc06ee40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 --2- 172.21.15.88:0/1202537802 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f38ec06c770 0x7f38ec155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.206+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 >> 172.21.15.88:0/1202537802 conn(0x7f38ec0b2d50 msgr2=0x7f38ec073500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:44.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.207+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 shutdown_connections 2024-04-26T20:02:44.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.208+0000 7f38f4200640 1 -- 172.21.15.88:0/1202537802 wait complete. 2024-04-26T20:02:44.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.412+0000 7f0be7e00640 1 Processor -- start 2024-04-26T20:02:44.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.412+0000 7f0be7e00640 1 -- start start 2024-04-26T20:02:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.413+0000 7f0be7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0be80d1580 0x7f0be8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.413+0000 7f0be7e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be80b5ff0 con 0x7f0be81675d0 2024-04-26T20:02:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.413+0000 7f0be7e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0be8167d40 con 0x7f0be80d03b0 2024-04-26T20:02:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.413+0000 7f0be7e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f0be8167a60 con 0x7f0be80d1580 2024-04-26T20:02:44.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0bde000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0be81675d0 0x7f0be80d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50980/0 (socket says 172.21.15.88:50980) 2024-04-26T20:02:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0bde000640 1 -- 172.21.15.88:0/3930844628 learned_addr learned my addr 172.21.15.88:0/3930844628 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875986555 0 0) 0x7f0be80b5ff0 con 0x7f0be81675d0 2024-04-26T20:02:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bcc003520 con 0x7f0be81675d0 2024-04-26T20:02:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be7400640 1 --2- 172.21.15.88:0/3930844628 >> v2:172.21.15.132:3300/0 conn(0x7f0be80d1580 0x7f0be8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872062324 0 0) 0x7f0bcc003520 con 0x7f0be81675d0 2024-04-26T20:02:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 >> v2:172.21.15.132:3300/0 conn(0x7f0be80d1580 msgr2=0x7f0be8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 --2- 172.21.15.88:0/3930844628 >> v2:172.21.15.132:3300/0 conn(0x7f0be80d1580 0x7f0be8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 >> v1:172.21.15.132:6789/0 conn(0x7f0be80d03b0 legacy=0x7f0be8166ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8163a00 con 0x7f0be81675d0 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f0bd0001670 con 0x7f0be81675d0 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.414+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 >> v1:172.21.15.88:6789/0 conn(0x7f0be81675d0 legacy=0x7f0be80d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0bcc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8167a60 con 0x7f0bcc0052e0 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc00c1b0 con 0x7f0bcc004800 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc00c3b0 con 0x7f0bcc0039b0 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0bcc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be6a00640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0bde000640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 -- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0bcc0039b0 msgr2=0x7f0bcc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0bcc0039b0 0x7f0bcc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 -- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bcc004800 msgr2=0x7f0bcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0be7400640 1 -- 172.21.15.88:0/3930844628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc003520 con 0x7f0bcc0052e0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.415+0000 7f0bde000640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0bcc004800 0x7f0bcc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.416+0000 7f0be7400640 1 -- 172.21.15.88:0/3930844628 reap_dead start 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.416+0000 7f0be7400640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0bcc00b9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007330 tx=0x7f0bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648b135abb1c9709 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0009be0 con 0x7f0bcc0052e0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0be0003b90 con 0x7f0bcc0052e0 2024-04-26T20:02:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be6000640 1 -- 172.21.15.88:0/3930844628 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0004020 con 0x7f0bcc0052e0 2024-04-26T20:02:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 -- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 msgr2=0x7f0bcc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0bcc00b9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007330 tx=0x7f0bcc005830 comp rx=0 tx=0).stop 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 -- 172.21.15.88:0/3930844628 shutdown_connections 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 --2- 172.21.15.88:0/3930844628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0bcc00b9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 -- 172.21.15.88:0/3930844628 >> 172.21.15.88:0/3930844628 conn(0x7f0be806c740 msgr2=0x7f0be806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 -- 172.21.15.88:0/3930844628 shutdown_connections 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.417+0000 7f0be7e00640 1 -- 172.21.15.88:0/3930844628 wait complete. 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.419+0000 7f0be7e00640 1 Processor -- start 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.419+0000 7f0be7e00640 1 -- start start 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be814fcf0 0x7f0be80b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be8150150 0x7f0be8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0be80d0790 con 0x7f0bcc0052e0 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8163fc0 con 0x7f0be8150150 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be7e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0be80d0910 con 0x7f0be814fcf0 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60658/0 (socket says 172.21.15.88:60658) 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.420+0000 7f0be6a00640 1 -- 172.21.15.88:0/53594877 learned_addr learned my addr 172.21.15.88:0/53594877 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 -- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be814fcf0 msgr2=0x7f0be80b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be814fcf0 0x7f0be80b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 -- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be8150150 msgr2=0x7f0be8154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be8150150 0x7f0be8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 -- 172.21.15.88:0/53594877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be80b0e60 con 0x7f0bcc0052e0 2024-04-26T20:02:44.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be6a00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800d5a0 tx=0x7f0bd800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8003610 con 0x7f0bcc0052e0 2024-04-26T20:02:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bd8017070 con 0x7f0bcc0052e0 2024-04-26T20:02:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80039f0 con 0x7f0bcc0052e0 2024-04-26T20:02:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.421+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be80b1140 con 0x7f0bcc0052e0 2024-04-26T20:02:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.422+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be80b1700 con 0x7f0bcc0052e0 2024-04-26T20:02:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.423+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be80693c0 con 0x7f0bcc0052e0 2024-04-26T20:02:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.424+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0bd8003b90 con 0x7f0bcc0052e0 2024-04-26T20:02:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.425+0000 7f0bdf400640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 0x7f0bc406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.425+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd808bb80 con 0x7f0bcc0052e0 2024-04-26T20:02:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.425+0000 7f0bde000640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 0x7f0bc406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.426+0000 7f0bde000640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 0x7f0bc406ee40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0be8166650 tx=0x7f0bd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.429+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0bd8059f00 con 0x7f0bcc0052e0 2024-04-26T20:02:44.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.628+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0be80b0b10 con 0x7f0bcc0052e0 2024-04-26T20:02:44.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.630+0000 7f0bdf400640 1 -- 172.21.15.88:0/53594877 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16669 (secure 0 0 0) 0x7f0bd8059800 con 0x7f0bcc0052e0 2024-04-26T20:02:44.631 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:44.631 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":62,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:43.648009+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 msgr2=0x7f0bc406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 0x7f0bc406ee40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0be8166650 tx=0x7f0bd0009040 comp rx=0 tx=0).stop 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 msgr2=0x7f0be8150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800d5a0 tx=0x7f0bd800da60 comp rx=0 tx=0).stop 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 shutdown_connections 2024-04-26T20:02:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0be8150150 0x7f0be8154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0bc406c980 0x7f0bc406ee40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0be814fcf0 0x7f0be80b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 --2- 172.21.15.88:0/53594877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bcc0052e0 0x7f0be8150c00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 >> 172.21.15.88:0/53594877 conn(0x7f0be806c740 msgr2=0x7f0be80730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 shutdown_connections 2024-04-26T20:02:44.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.633+0000 7f0be7e00640 1 -- 172.21.15.88:0/53594877 wait complete. 2024-04-26T20:02:44.652 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:44.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.840+0000 7f88cf000640 1 Processor -- start 2024-04-26T20:02:44.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.840+0000 7f88cf000640 1 -- start start 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88cf000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f88c815f380 0x7f88c806f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88cf000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c80cb5e0 con 0x7f88c806fe40 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88cf000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c80703a0 con 0x7f88c8073000 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88cf000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f88c80cb860 con 0x7f88c815f380 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88c7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f88c815f380 0x7f88c806f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88c7e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f88c815f380 0x7f88c806f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60668/0 (socket says 172.21.15.88:60668) 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88c7e00640 1 -- 172.21.15.88:0/8697828 learned_addr learned my addr 172.21.15.88:0/8697828 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:44.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.842+0000 7f88c7400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f88c806fe40 0x7f88c80728f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:50986/0 (socket says 172.21.15.88:50986) 2024-04-26T20:02:44.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.843+0000 7f88c7e00640 1 -- 172.21.15.88:0/8697828 >> v1:172.21.15.132:6789/0 conn(0x7f88c8073000 legacy=0x7f88c80cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:44.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.843+0000 7f88c7e00640 1 -- 172.21.15.88:0/8697828 >> v1:172.21.15.88:6789/0 conn(0x7f88c806fe40 legacy=0x7f88c80728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.843+0000 7f88c7e00640 1 -- 172.21.15.88:0/8697828 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c80cbb40 con 0x7f88c815f380 2024-04-26T20:02:44.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.844+0000 7f88c7e00640 1 --2- 172.21.15.88:0/8697828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c815f380 0x7f88c806f900 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009e50 tx=0x7f88b802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77755d9796d35712 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.844+0000 7f88c6a00640 1 -- 172.21.15.88:0/8697828 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8036070 con 0x7f88c815f380 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.844+0000 7f88c6a00640 1 -- 172.21.15.88:0/8697828 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b8031600 con 0x7f88c815f380 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.844+0000 7f88c6a00640 1 -- 172.21.15.88:0/8697828 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80319d0 con 0x7f88c815f380 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 -- 172.21.15.88:0/8697828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c815f380 msgr2=0x7f88c806f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 --2- 172.21.15.88:0/8697828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c815f380 0x7f88c806f900 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009e50 tx=0x7f88b802a6d0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 -- 172.21.15.88:0/8697828 shutdown_connections 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 --2- 172.21.15.88:0/8697828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c815f380 0x7f88c806f900 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 -- 172.21.15.88:0/8697828 >> 172.21.15.88:0/8697828 conn(0x7f88c8070b60 msgr2=0x7f88c806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 -- 172.21.15.88:0/8697828 shutdown_connections 2024-04-26T20:02:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.845+0000 7f88cf000640 1 -- 172.21.15.88:0/8697828 wait complete. 2024-04-26T20:02:44.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.847+0000 7f88cf000640 1 Processor -- start 2024-04-26T20:02:44.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.847+0000 7f88cf000640 1 -- start start 2024-04-26T20:02:44.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88c806fe40 0x7f88c815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88c815f380 0x7f88c8147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f88c806ed10 con 0x7f88c8073000 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f88c806eb90 con 0x7f88c815f380 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88cf000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f88c80cca50 con 0x7f88c806fe40 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88c7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88c7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60672/0 (socket says 172.21.15.88:60672) 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.848+0000 7f88c7400640 1 -- 172.21.15.88:0/3328304314 learned_addr learned my addr 172.21.15.88:0/3328304314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 -- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88c806fe40 msgr2=0x7f88c815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88c806fe40 0x7f88c815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 -- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88c815f380 msgr2=0x7f88c8147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88c815f380 0x7f88c8147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 -- 172.21.15.88:0/3328304314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81e9b40 con 0x7f88c8073000 2024-04-26T20:02:44.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88c7400640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c400 tx=0x7f88b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00074e0 con 0x7f88c8073000 2024-04-26T20:02:44.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b0007680 con 0x7f88c8073000 2024-04-26T20:02:44.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81e8aa0 con 0x7f88c8073000 2024-04-26T20:02:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0004a00 con 0x7f88c8073000 2024-04-26T20:02:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.849+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81e9060 con 0x7f88c8073000 2024-04-26T20:02:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.850+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c8069540 con 0x7f88c8073000 2024-04-26T20:02:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.851+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f88b001a020 con 0x7f88c8073000 2024-04-26T20:02:44.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.852+0000 7f88bfe00640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 0x7f88a406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:44.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.852+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f88b008b510 con 0x7f88c8073000 2024-04-26T20:02:44.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.852+0000 7f88c7e00640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 0x7f88a406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:44.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.853+0000 7f88c7e00640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 0x7f88a406ee40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88b8033000 tx=0x7f88b802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:44.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:44.857+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88c80cb5e0 con 0x7f88c8073000 2024-04-26T20:02:45.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.060+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88c80cdd90 con 0x7f88c8073000 2024-04-26T20:02:45.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.063+0000 7f88bfe00640 1 -- 172.21.15.88:0/3328304314 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16669 (secure 0 0 0) 0x7f88b0010300 con 0x7f88c8073000 2024-04-26T20:02:45.064 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:45.065 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":62,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:43.648009+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:45.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.066+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 msgr2=0x7f88a406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.066+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 0x7f88a406ee40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88b8033000 tx=0x7f88b802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:45.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.066+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 msgr2=0x7f88c81e8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.066+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c400 tx=0x7f88b000c8c0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 shutdown_connections 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88c815f380 0x7f88c8147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c8073000 0x7f88c81e8330 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f88a406c980 0x7f88a406ee40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 --2- 172.21.15.88:0/3328304314 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f88c806fe40 0x7f88c815b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 >> 172.21.15.88:0/3328304314 conn(0x7f88c8070b60 msgr2=0x7f88c80b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 shutdown_connections 2024-04-26T20:02:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.067+0000 7f88cf000640 1 -- 172.21.15.88:0/3328304314 wait complete. 2024-04-26T20:02:45.087 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:02:45.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.283+0000 7fc63b400640 1 Processor -- start 2024-04-26T20:02:45.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.284+0000 7fc63b400640 1 -- start start 2024-04-26T20:02:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc63b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 0x7fc63415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc63b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc63406c770 0x7fc63406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc63b400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc63406eea0 con 0x7fc634073120 2024-04-26T20:02:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc63b400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc6340b4c40 con 0x7fc63406c770 2024-04-26T20:02:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc63b400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc63406f140 con 0x7fc634156650 2024-04-26T20:02:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc639400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc634073120 0x7fc6340735b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51000/0 (socket says 172.21.15.88:51000) 2024-04-26T20:02:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.285+0000 7fc639400640 1 -- 172.21.15.88:0/4122667241 learned_addr learned my addr 172.21.15.88:0/4122667241 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:45.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775660380 0 0) 0x7fc63406eea0 con 0x7fc634073120 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc638a00640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 0x7fc63415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633e00640 1 --2- 172.21.15.88:0/4122667241 >> v2:172.21.15.132:3300/0 conn(0x7fc63406c770 0x7fc63406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc618005730 con 0x7fc634073120 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1217395768 0 0) 0x7fc618005730 con 0x7fc634073120 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 msgr2=0x7fc63415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 0x7fc63415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 >> v2:172.21.15.132:3300/0 conn(0x7fc63406c770 msgr2=0x7fc63406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 --2- 172.21.15.88:0/4122667241 >> v2:172.21.15.132:3300/0 conn(0x7fc63406c770 0x7fc63406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63406f420 con 0x7fc634073120 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fc62c001670 con 0x7fc634073120 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 >> v1:172.21.15.88:6789/0 conn(0x7fc634073120 legacy=0x7fc6340735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc618005bc0 0x7fc618005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc618006a10 0x7fc618006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 0x7fc6180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340b4c40 con 0x7fc618005bc0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406f140 con 0x7fc618006a10 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.286+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc61800dc20 con 0x7fc634156650 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc618005bc0 0x7fc618005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 -- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 msgr2=0x7fc6180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc634156650 0x7fc6180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 -- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc618006a10 msgr2=0x7fc618006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc618006a10 0x7fc618006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.287+0000 7fc638a00640 1 -- 172.21.15.88:0/4122667241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc618005730 con 0x7fc618005bc0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.288+0000 7fc638a00640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc618005bc0 0x7fc618005fe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc62400ba30 tx=0x7fc618007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c07551f9ab8bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.288+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624030070 con 0x7fc618005bc0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.288+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62402ae10 con 0x7fc618005bc0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.288+0000 7fc633400640 1 -- 172.21.15.88:0/4122667241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62400b600 con 0x7fc618005bc0 2024-04-26T20:02:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 -- 172.21.15.88:0/4122667241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc618005bc0 msgr2=0x7fc618005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 --2- 172.21.15.88:0/4122667241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc618005bc0 0x7fc618005fe0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc62400ba30 tx=0x7fc618007a40 comp rx=0 tx=0).stop 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc639400640 1 -- 172.21.15.88:0/4122667241 reap_dead start 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 -- 172.21.15.88:0/4122667241 shutdown_connections 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 -- 172.21.15.88:0/4122667241 >> 172.21.15.88:0/4122667241 conn(0x7fc6340b2d50 msgr2=0x7fc6340b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 -- 172.21.15.88:0/4122667241 shutdown_connections 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.289+0000 7fc63b400640 1 -- 172.21.15.88:0/4122667241 wait complete. 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.290+0000 7fc63b400640 1 Processor -- start 2024-04-26T20:02:45.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.291+0000 7fc63b400640 1 -- start start 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.291+0000 7fc63b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 0x7fc6341f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc63b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc63b400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 0x7fc63409c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc63b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406fcc0 con 0x7fc6340d0cc0 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc63b400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406f9e0 con 0x7fc63409bcf0 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc63b400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc63415c680 con 0x7fc618005bc0 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60678/0 (socket says 172.21.15.88:60678) 2024-04-26T20:02:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 -- 172.21.15.88:0/1346590704 learned_addr learned my addr 172.21.15.88:0/1346590704 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc638a00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 0x7fc6341f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc639400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 0x7fc63409c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 -- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 msgr2=0x7fc6341f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 0x7fc6341f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 -- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 msgr2=0x7fc63409c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 0x7fc63409c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc633e00640 1 -- 172.21.15.88:0/1346590704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63409c6b0 con 0x7fc6340d0cc0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc638a00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 0x7fc6341f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.292+0000 7fc639400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 0x7fc63409c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc633e00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc620009db0 tx=0x7fc62000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620018020 con 0x7fc6340d0cc0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc620014480 con 0x7fc6340d0cc0 2024-04-26T20:02:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620014870 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63409d550 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.293+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63409cc20 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.294+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc634069650 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.295+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc620005d10 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.296+0000 7fc630c00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 0x7fc61406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.296+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fc62008c740 con 0x7fc6340d0cc0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.296+0000 7fc638a00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 0x7fc61406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:45.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.297+0000 7fc638a00640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 0x7fc61406ee40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc62400ba00 tx=0x7fc624030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.300+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc620022070 con 0x7fc6340d0cc0 2024-04-26T20:02:45.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:45.527+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc6341f6230 con 0x7fc6340d0cc0 2024-04-26T20:02:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.211+0000 7fc630c00640 1 -- 172.21.15.88:0/1346590704 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) ==== 118+0+0 (secure 0 0 0) 0x7fc620010040 con 0x7fc6340d0cc0 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 msgr2=0x7fc61406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 0x7fc61406ee40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc62400ba00 tx=0x7fc624030000 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 msgr2=0x7fc6340d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc620009db0 tx=0x7fc62000c490 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 shutdown_connections 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc63409bcf0 0x7fc63409c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6340d0cc0 0x7fc6340d1140 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc61406c980 0x7fc61406ee40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 --2- 172.21.15.88:0/1346590704 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc618005bc0 0x7fc6341f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.215+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 >> 172.21.15.88:0/1346590704 conn(0x7fc6340b2d50 msgr2=0x7fc6341509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:46.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.216+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 shutdown_connections 2024-04-26T20:02:46.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.216+0000 7fc63b400640 1 -- 172.21.15.88:0/1346590704 wait complete. 2024-04-26T20:02:46.239 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:02:46.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:02:46.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.437+0000 7f4277200640 1 Processor -- start 2024-04-26T20:02:46.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.437+0000 7f4277200640 1 -- start start 2024-04-26T20:02:46.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4278071e80 0x7f42780b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42780b4ed0 0x7f42780ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4278072260 0x7f42780726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f427806e320 con 0x7f4278071e80 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f427806e600 con 0x7f4278072260 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.439+0000 7f4277200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f427806e8e0 con 0x7f42780b4ed0 2024-04-26T20:02:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4278071e80 0x7f42780b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f4278071e80 0x7f42780b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60694/0 (socket says 172.21.15.88:60694) 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 -- 172.21.15.88:0/955756164 learned_addr learned my addr 172.21.15.88:0/955756164 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4276800640 1 --2- 172.21.15.88:0/955756164 >> v2:172.21.15.132:3300/0 conn(0x7f4278072260 0x7f42780726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f426fe00640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42780b4ed0 0x7f42780ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 -- 172.21.15.88:0/955756164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42780b4ed0 msgr2=0x7f42780ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:46.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42780b4ed0 0x7f42780ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 -- 172.21.15.88:0/955756164 >> v2:172.21.15.132:3300/0 conn(0x7f4278072260 msgr2=0x7f42780726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:46.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 --2- 172.21.15.88:0/955756164 >> v2:172.21.15.132:3300/0 conn(0x7f4278072260 0x7f42780726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.440+0000 7f4275e00640 1 -- 172.21.15.88:0/955756164 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427806ebc0 con 0x7f4278071e80 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.441+0000 7f4275e00640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4278071e80 0x7f42780b3ee0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4278073d50 tx=0x7f426802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8633d2c1537612 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4275400640 1 -- 172.21.15.88:0/955756164 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268037070 con 0x7f4278071e80 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4275400640 1 -- 172.21.15.88:0/955756164 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4268032440 con 0x7f4278071e80 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4275400640 1 -- 172.21.15.88:0/955756164 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268032810 con 0x7f4278071e80 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 -- 172.21.15.88:0/955756164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4278071e80 msgr2=0x7f42780b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4278071e80 0x7f42780b3ee0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4278073d50 tx=0x7f426802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 -- 172.21.15.88:0/955756164 shutdown_connections 2024-04-26T20:02:46.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 --2- 172.21.15.88:0/955756164 >> v2:172.21.15.132:3300/0 conn(0x7f4278072260 0x7f42780726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f42780b4ed0 0x7f42780ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 --2- 172.21.15.88:0/955756164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4278071e80 0x7f42780b3ee0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.442+0000 7f4277200640 1 -- 172.21.15.88:0/955756164 >> 172.21.15.88:0/955756164 conn(0x7f42780b2900 msgr2=0x7f42780b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.443+0000 7f4277200640 1 -- 172.21.15.88:0/955756164 shutdown_connections 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.443+0000 7f4277200640 1 -- 172.21.15.88:0/955756164 wait complete. 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.444+0000 7f4277200640 1 Processor -- start 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.444+0000 7f4277200640 1 -- start start 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4278071e80 0x7f42780cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4278072260 0x7f42780ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b5ff0 con 0x7f42780b4ed0 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f427815f4d0 con 0x7f4278072260 2024-04-26T20:02:46.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.445+0000 7f4277200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f427806f0e0 con 0x7f4278071e80 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60696/0 (socket says 172.21.15.88:60696) 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 -- 172.21.15.88:0/3662484151 learned_addr learned my addr 172.21.15.88:0/3662484151 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4275e00640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4278071e80 0x7f42780cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f426fe00640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4278072260 0x7f42780ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 -- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4278071e80 msgr2=0x7f42780cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4278071e80 0x7f42780cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 -- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4278072260 msgr2=0x7f42780ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4278072260 0x7f42780ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:46.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.446+0000 7f4276800640 1 -- 172.21.15.88:0/3662484151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42780cea30 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f4276800640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4270007d40 tx=0x7f427000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270014420 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4270019070 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270014820 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42780cecd0 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.447+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42780cf290 con 0x7f42780b4ed0 2024-04-26T20:02:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.448+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42780693c0 con 0x7f42780b4ed0 2024-04-26T20:02:46.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.449+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f42700149c0 con 0x7f42780b4ed0 2024-04-26T20:02:46.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.450+0000 7f426e000640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 0x7f425406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:46.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.450+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f427008c360 con 0x7f42780b4ed0 2024-04-26T20:02:46.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.450+0000 7f4275e00640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 0x7f425406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:46.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.451+0000 7f4275e00640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 0x7f425406ee70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4268034000 tx=0x7f426802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:46.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.454+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f427005a730 con 0x7f42780b4ed0 2024-04-26T20:02:46.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:46.697+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4278155af0 con 0x7f42780b4ed0 2024-04-26T20:02:47.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.222+0000 7f426e000640 1 -- 172.21.15.88:0/3662484151 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7f427005a030 con 0x7f42780b4ed0 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 msgr2=0x7f425406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 0x7f425406ee70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4268034000 tx=0x7f426802e000 comp rx=0 tx=0).stop 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 msgr2=0x7f4278140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4270007d40 tx=0x7f427000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 shutdown_connections 2024-04-26T20:02:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42780b4ed0 0x7f4278140fc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4278072260 0x7f42780ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f425406c9b0 0x7f425406ee70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 --2- 172.21.15.88:0/3662484151 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4278071e80 0x7f42780cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.226+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 >> 172.21.15.88:0/3662484151 conn(0x7f42780b2900 msgr2=0x7f427815e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.227+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 shutdown_connections 2024-04-26T20:02:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.227+0000 7f4277200640 1 -- 172.21.15.88:0/3662484151 wait complete. 2024-04-26T20:02:47.250 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:02:47.250 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:47.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.447+0000 7fef8d800640 1 Processor -- start 2024-04-26T20:02:47.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.447+0000 7fef8d800640 1 -- start start 2024-04-26T20:02:47.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef88163210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8d800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef880d3f50 con 0x7fef880b82b0 2024-04-26T20:02:47.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8d800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef880d36b0 con 0x7fef8815f1b0 2024-04-26T20:02:47.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fef880d3990 con 0x7fef880d5990 2024-04-26T20:02:47.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8ce00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fef880b82b0 0x7fef880b8740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51008/0 (socket says 172.21.15.88:51008) 2024-04-26T20:02:47.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef8ce00640 1 -- 172.21.15.88:0/2775545459 learned_addr learned my addr 172.21.15.88:0/2775545459 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef87400640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef88163210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.449+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2024214178 0 0) 0x7fef880d3f50 con 0x7fef880b82b0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef6c003520 con 0x7fef880b82b0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3482344675 0 0) 0x7fef6c003520 con 0x7fef880b82b0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 msgr2=0x7fef88163210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef88163210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 >> v1:172.21.15.132:6789/0 conn(0x7fef8815f1b0 legacy=0x7fef88162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef880d3c70 con 0x7fef880b82b0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fef80001670 con 0x7fef880b82b0 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 >> v1:172.21.15.88:6789/0 conn(0x7fef880b82b0 legacy=0x7fef880b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c004800 0x7fef6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef880d3990 con 0x7fef6c0039e0 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c00ba70 con 0x7fef6c004800 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c00bc70 con 0x7fef880d5990 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.450+0000 7fef87e00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87e00640 1 -- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 msgr2=0x7fef6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef8ce00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87400640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c004800 0x7fef6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87e00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87e00640 1 -- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c004800 msgr2=0x7fef6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87e00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef6c004800 0x7fef6c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef87e00640 1 -- 172.21.15.88:0/2775545459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c003520 con 0x7fef6c0039e0 2024-04-26T20:02:47.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.451+0000 7fef8ce00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880d5990 0x7fef6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.452+0000 7fef87e00640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef6c003e00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fef78001720 tx=0x7fef6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3fcd3b70777663 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.452+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78001370 con 0x7fef6c0039e0 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.452+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef780052d0 con 0x7fef6c0039e0 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.452+0000 7fef86a00640 1 -- 172.21.15.88:0/2775545459 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7802ab90 con 0x7fef6c0039e0 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 -- 172.21.15.88:0/2775545459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 msgr2=0x7fef6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 --2- 172.21.15.88:0/2775545459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef6c003e00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fef78001720 tx=0x7fef6c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8ce00640 1 -- 172.21.15.88:0/2775545459 reap_dead start 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 -- 172.21.15.88:0/2775545459 shutdown_connections 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 -- 172.21.15.88:0/2775545459 >> 172.21.15.88:0/2775545459 conn(0x7fef8809e230 msgr2=0x7fef880d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 -- 172.21.15.88:0/2775545459 shutdown_connections 2024-04-26T20:02:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.453+0000 7fef8d800640 1 -- 172.21.15.88:0/2775545459 wait complete. 2024-04-26T20:02:47.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.455+0000 7fef8d800640 1 Processor -- start 2024-04-26T20:02:47.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.455+0000 7fef8d800640 1 -- start start 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880ce7c0 0x7fef881f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 0x7fef8814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef88163210 con 0x7fef6c0039e0 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fef880d33a0 con 0x7fef88149c80 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef8d800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fef88163390 con 0x7fef880ce7c0 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef87e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef87e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60716/0 (socket says 172.21.15.88:60716) 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.456+0000 7fef87e00640 1 -- 172.21.15.88:0/3275812650 learned_addr learned my addr 172.21.15.88:0/3275812650 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef8ce00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 0x7fef8814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87400640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880ce7c0 0x7fef881f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 -- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880ce7c0 msgr2=0x7fef881f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880ce7c0 0x7fef881f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 -- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 msgr2=0x7fef8814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 0x7fef8814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 -- 172.21.15.88:0/3275812650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef880cec80 con 0x7fef6c0039e0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef8ce00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 0x7fef8814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef87e00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fef78001840 tx=0x7fef78001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78037040 con 0x7fef6c0039e0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef78005480 con 0x7fef6c0039e0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78007a30 con 0x7fef6c0039e0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.457+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef881601f0 con 0x7fef6c0039e0 2024-04-26T20:02:47.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.458+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef881607b0 con 0x7fef6c0039e0 2024-04-26T20:02:47.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.459+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef880693c0 con 0x7fef6c0039e0 2024-04-26T20:02:47.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.460+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fef78044020 con 0x7fef6c0039e0 2024-04-26T20:02:47.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.461+0000 7fef7fe00640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 0x7fef6806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.461+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fef780ab8a0 con 0x7fef6c0039e0 2024-04-26T20:02:47.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.461+0000 7fef87400640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 0x7fef6806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.462+0000 7fef87400640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 0x7fef6806ee40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef880bac30 tx=0x7fef7400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.465+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef78079c70 con 0x7fef6c0039e0 2024-04-26T20:02:47.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.704+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef8808d090 con 0x7fef6c0039e0 2024-04-26T20:02:47.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.705+0000 7fef7fe00640 1 -- 172.21.15.88:0/3275812650 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5186 (secure 0 0 0) 0x7fef78079570 con 0x7fef6c0039e0 2024-04-26T20:02:47.706 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:47.706 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:02:47.706 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 39 2024-04-26T20:02:47.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 msgr2=0x7fef6806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 0x7fef6806ee40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fef880bac30 tx=0x7fef7400b040 comp rx=0 tx=0).stop 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 msgr2=0x7fef881f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fef78001840 tx=0x7fef78001120 comp rx=0 tx=0).stop 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 shutdown_connections 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef88149c80 0x7fef8814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fef6806c980 0x7fef6806ee40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fef880ce7c0 0x7fef881f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 --2- 172.21.15.88:0/3275812650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef6c0039e0 0x7fef881f4760 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 >> 172.21.15.88:0/3275812650 conn(0x7fef8809e230 msgr2=0x7fef880d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:47.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 shutdown_connections 2024-04-26T20:02:47.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.708+0000 7fef8d800640 1 -- 172.21.15.88:0/3275812650 wait complete. 2024-04-26T20:02:47.728 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:02:47.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.929+0000 7f70eb200640 1 Processor -- start 2024-04-26T20:02:47.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.929+0000 7f70eb200640 1 -- start start 2024-04-26T20:02:47.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.930+0000 7f70eb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70e40b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.930+0000 7f70eb200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f70e40d0e70 0x7f70e40d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70eb200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e4146a60 con 0x7f70e4160930 2024-04-26T20:02:47.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70eb200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f70e406e970 con 0x7f70e40d0e70 2024-04-26T20:02:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70eb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40d17f0 con 0x7f70e4146680 2024-04-26T20:02:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70ea800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f70e4160930 0x7f70e4160dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:51014/0 (socket says 172.21.15.88:51014) 2024-04-26T20:02:47.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70ea800640 1 -- 172.21.15.88:0/1473650712 learned_addr learned my addr 172.21.15.88:0/1473650712 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70e9400640 1 --2- 172.21.15.88:0/1473650712 >> v2:172.21.15.132:3300/0 conn(0x7f70e40d0e70 0x7f70e40d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70e9e00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70e40b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731595972 0 0) 0x7f70e4146a60 con 0x7f70e4160930 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.931+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70cc003520 con 0x7f70e4160930 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3380083444 0 0) 0x7f70cc003520 con 0x7f70e4160930 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 msgr2=0x7f70e40b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70e40b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 >> v2:172.21.15.132:3300/0 conn(0x7f70e40d0e70 msgr2=0x7f70e40d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 --2- 172.21.15.88:0/1473650712 >> v2:172.21.15.132:3300/0 conn(0x7f70e40d0e70 0x7f70e40d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e406dd60 con 0x7f70e4160930 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f70e00019f0 con 0x7f70e4160930 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 >> v1:172.21.15.88:6789/0 conn(0x7f70e4160930 legacy=0x7f70e4160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70cc004d40 0x7f70cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d17f0 con 0x7f70cc004d40 2024-04-26T20:02:47.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc00b9c0 con 0x7f70cc0039b0 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc00bb40 con 0x7f70e4146680 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.932+0000 7f70ea800640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70cc004d40 0x7f70cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70e9400640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70e9e00640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70ea800640 1 -- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 msgr2=0x7f70cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70ea800640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4146680 0x7f70cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70ea800640 1 -- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70cc0039b0 msgr2=0x7f70cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70ea800640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.933+0000 7f70ea800640 1 -- 172.21.15.88:0/1473650712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc003520 con 0x7f70cc004d40 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.934+0000 7f70ea800640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70cc004d40 0x7f70cc0051c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a7a0 tx=0x7f70cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594ba5a405a22f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.934+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000a3f0 con 0x7f70cc004d40 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.934+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e002f070 con 0x7f70cc004d40 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.934+0000 7f70e8a00640 1 -- 172.21.15.88:0/1473650712 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e002bc20 con 0x7f70cc004d40 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 -- 172.21.15.88:0/1473650712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70cc004d40 msgr2=0x7f70cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 --2- 172.21.15.88:0/1473650712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70cc004d40 0x7f70cc0051c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a7a0 tx=0x7f70cc005830 comp rx=0 tx=0).stop 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70ea800640 1 -- 172.21.15.88:0/1473650712 reap_dead start 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 -- 172.21.15.88:0/1473650712 shutdown_connections 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 -- 172.21.15.88:0/1473650712 >> 172.21.15.88:0/1473650712 conn(0x7f70e406c740 msgr2=0x7f70e406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 -- 172.21.15.88:0/1473650712 shutdown_connections 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.935+0000 7f70eb200640 1 -- 172.21.15.88:0/1473650712 wait complete. 2024-04-26T20:02:47.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.937+0000 7f70eb200640 1 Processor -- start 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.937+0000 7f70eb200640 1 -- start start 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e4073760 0x7f70e40cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4073b40 0x7f70e40c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70e406e600 con 0x7f70e40ca3e0 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f70e406e320 con 0x7f70e4073760 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70eb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40cd6a0 con 0x7f70e4073b40 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60730/0 (socket says 172.21.15.88:60730) 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 -- 172.21.15.88:0/999064391 learned_addr learned my addr 172.21.15.88:0/999064391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:47.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70e9e00640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e4073760 0x7f70e40cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 -- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4073b40 msgr2=0x7f70e40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70e9400640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4073b40 0x7f70e40c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4073b40 0x7f70e40c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 -- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e4073760 msgr2=0x7f70e40cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e4073760 0x7f70e40cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.938+0000 7f70ea800640 1 -- 172.21.15.88:0/999064391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e40cb970 con 0x7f70e40ca3e0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70ea800640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a8c0 tx=0x7f70e0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000a3f0 con 0x7f70e40ca3e0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e00019e0 con 0x7f70e40ca3e0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0001f60 con 0x7f70e40ca3e0 2024-04-26T20:02:47.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e41f8130 con 0x7f70e40ca3e0 2024-04-26T20:02:47.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.939+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41f86f0 con 0x7f70e40ca3e0 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.940+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e40693c0 con 0x7f70e40ca3e0 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.941+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f70e0034670 con 0x7f70e40ca3e0 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.942+0000 7f70de000640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 0x7f70c806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.942+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f70e00ab6b0 con 0x7f70e40ca3e0 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.942+0000 7f70e9e00640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 0x7f70c806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:47.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.943+0000 7f70e9e00640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 0x7f70c806ee70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009990 tx=0x7f70d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:47.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:47.947+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70e0047030 con 0x7f70e40ca3e0 2024-04-26T20:02:48.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.148+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e40008d0 con 0x7f70e40ca3e0 2024-04-26T20:02:48.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.151+0000 7f70de000640 1 -- 172.21.15.88:0/999064391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16656 (secure 0 0 0) 0x7f70e0079af0 con 0x7f70e40ca3e0 2024-04-26T20:02:48.153 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:48.153 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":63,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:02:45.199045+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 msgr2=0x7f70c806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 0x7f70c806ee70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009990 tx=0x7f70d8008040 comp rx=0 tx=0).stop 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 msgr2=0x7f70e40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a8c0 tx=0x7f70e0001360 comp rx=0 tx=0).stop 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 shutdown_connections 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e40ca3e0 0x7f70e40cfc40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f70c806c9b0 0x7f70c806ee70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f70e4073b40 0x7f70e40c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 --2- 172.21.15.88:0/999064391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e4073760 0x7f70e40cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 >> 172.21.15.88:0/999064391 conn(0x7f70e406c740 msgr2=0x7f70e40ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 shutdown_connections 2024-04-26T20:02:48.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.154+0000 7f70eb200640 1 -- 172.21.15.88:0/999064391 wait complete. 2024-04-26T20:02:48.173 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:02:48.173 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:48.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.377+0000 7f1f52e00640 1 Processor -- start 2024-04-26T20:02:48.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.377+0000 7f1f52e00640 1 -- start start 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f52e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f4c15cb70 0x7f1f4c06ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f52e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f52e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4c0ae850 con 0x7f1f4c06c770 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f52e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4c0af460 con 0x7f1f4c15cb70 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f52e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0aeb30 con 0x7f1f4c15cfd0 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f4b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f4c15cb70 0x7f1f4c06ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f4b400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1f4c15cb70 0x7f1f4c06ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60732/0 (socket says 172.21.15.88:60732) 2024-04-26T20:02:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.379+0000 7f1f4b400640 1 -- 172.21.15.88:0/3231769388 learned_addr learned my addr 172.21.15.88:0/3231769388 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.380+0000 7f1f4b400640 1 -- 172.21.15.88:0/3231769388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 msgr2=0x7f1f4c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.380+0000 7f1f50e00640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.380+0000 7f1f4b400640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.380+0000 7f1f4b400640 1 -- 172.21.15.88:0/3231769388 >> v1:172.21.15.132:6789/0 conn(0x7f1f4c06c770 legacy=0x7f1f4c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.380+0000 7f1f4b400640 1 -- 172.21.15.88:0/3231769388 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c0aee10 con 0x7f1f4c15cb70 2024-04-26T20:02:48.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.381+0000 7f1f4b400640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c06ee60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bed1f47898d5186 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.381+0000 7f1f4aa00640 1 -- 172.21.15.88:0/3231769388 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38037070 con 0x7f1f4c15cb70 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.381+0000 7f1f4aa00640 1 -- 172.21.15.88:0/3231769388 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f38032440 con 0x7f1f4c15cb70 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.381+0000 7f1f4aa00640 1 -- 172.21.15.88:0/3231769388 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38032810 con 0x7f1f4c15cb70 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 -- 172.21.15.88:0/3231769388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 msgr2=0x7f1f4c06ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c06ee60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3802a200 comp rx=0 tx=0).stop 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 -- 172.21.15.88:0/3231769388 shutdown_connections 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c06f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 --2- 172.21.15.88:0/3231769388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c06ee60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 -- 172.21.15.88:0/3231769388 >> 172.21.15.88:0/3231769388 conn(0x7f1f4c0b2d50 msgr2=0x7f1f4c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 -- 172.21.15.88:0/3231769388 shutdown_connections 2024-04-26T20:02:48.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.382+0000 7f1f52e00640 1 -- 172.21.15.88:0/3231769388 wait complete. 2024-04-26T20:02:48.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.384+0000 7f1f52e00640 1 Processor -- start 2024-04-26T20:02:48.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.384+0000 7f1f52e00640 1 -- start start 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c06c770 0x7f1f4c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c15c780 con 0x7f1f4c15cb70 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c15c600 con 0x7f1f4c06c770 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f52e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4c0af8f0 con 0x7f1f4c15cfd0 2024-04-26T20:02:48.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f4b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f4b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60744/0 (socket says 172.21.15.88:60744) 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f4b400640 1 -- 172.21.15.88:0/886845009 learned_addr learned my addr 172.21.15.88:0/886845009 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.385+0000 7f1f4be00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c06c770 0x7f1f4c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 -- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 msgr2=0x7f1f4c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f50e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 -- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c06c770 msgr2=0x7f1f4c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c06c770 0x7f1f4c0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 -- 172.21.15.88:0/886845009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c0ca640 con 0x7f1f4c15cb70 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f4b400640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38041020 con 0x7f1f4c15cb70 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f38030970 con 0x7f1f4c15cb70 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c0ca920 con 0x7f1f4c15cb70 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.386+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38037040 con 0x7f1f4c15cb70 2024-04-26T20:02:48.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.387+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c0caee0 con 0x7f1f4c15cb70 2024-04-26T20:02:48.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.388+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c09ee40 con 0x7f1f4c15cb70 2024-04-26T20:02:48.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.389+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1f38044020 con 0x7f1f4c15cb70 2024-04-26T20:02:48.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.390+0000 7f1f43e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.390+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f1f380ac6a0 con 0x7f1f4c15cb70 2024-04-26T20:02:48.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.390+0000 7f1f4be00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.391+0000 7f1f4be00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 0x7f1f2c06ee40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c004ea0 tx=0x7f1f3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.394+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f3804b030 con 0x7f1f4c15cb70 2024-04-26T20:02:48.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.591+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f4c0cd430 con 0x7f1f2c06c980 2024-04-26T20:02:48.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.610+0000 7f1f43e00640 1 -- 172.21.15.88:0/886845009 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251999 (secure 0 0 0) 0x7f1f4c0cd430 con 0x7f1f2c06c980 2024-04-26T20:02:48.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:48.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.615+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 msgr2=0x7f1f2c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.615+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 0x7f1f2c06ee40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c004ea0 tx=0x7f1f3c009040 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.615+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 msgr2=0x7f1f4c0cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.615+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f380046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 shutdown_connections 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1f2c06c980 0x7f1f2c06ee40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1f4c15cfd0 0x7f1f4c15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f4c15cb70 0x7f1f4c0cb6e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 --2- 172.21.15.88:0/886845009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f4c06c770 0x7f1f4c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 >> 172.21.15.88:0/886845009 conn(0x7f1f4c0b2d50 msgr2=0x7f1f4c160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 shutdown_connections 2024-04-26T20:02:48.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.616+0000 7f1f52e00640 1 -- 172.21.15.88:0/886845009 wait complete. 2024-04-26T20:02:48.635 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-26T20:02:47.006821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":7092,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":7262208,"data_stored":3753229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.851667"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242179+0000","last_change":"2024-04-26T20:02:44.231595+0000","last_active":"2024-04-26T20:02:44.242179+0000","last_peered":"2024-04-26T20:02:44.242179+0000","last_clean":"2024-04-26T20:02:44.242179+0000","last_became_active":"2024-04-26T20:02:44.227391+0000","last_became_peered":"2024-04-26T20:02:44.227391+0000","last_unstale":"2024-04-26T20:02:44.242179+0000","last_undegraded":"2024-04-26T20:02:44.242179+0000","last_fullsized":"2024-04-26T20:02:44.242179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:38.763953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224039+0000","last_change":"2024-04-26T20:02:44.216625+0000","last_active":"2024-04-26T20:02:44.224039+0000","last_peered":"2024-04-26T20:02:44.224039+0000","last_clean":"2024-04-26T20:02:44.224039+0000","last_became_active":"2024-04-26T20:02:44.215725+0000","last_became_peered":"2024-04-26T20:02:44.215725+0000","last_unstale":"2024-04-26T20:02:44.224039+0000","last_undegraded":"2024-04-26T20:02:44.224039+0000","last_fullsized":"2024-04-26T20:02:44.224039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:36.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224457+0000","last_change":"2024-04-26T20:02:44.213989+0000","last_active":"2024-04-26T20:02:44.224457+0000","last_peered":"2024-04-26T20:02:44.224457+0000","last_clean":"2024-04-26T20:02:44.224457+0000","last_became_active":"2024-04-26T20:02:44.213685+0000","last_became_peered":"2024-04-26T20:02:44.213685+0000","last_unstale":"2024-04-26T20:02:44.224457+0000","last_undegraded":"2024-04-26T20:02:44.224457+0000","last_fullsized":"2024-04-26T20:02:44.224457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:03.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205699+0000","last_change":"2024-04-26T20:02:44.202223+0000","last_active":"2024-04-26T20:02:44.205699+0000","last_peered":"2024-04-26T20:02:44.205699+0000","last_clean":"2024-04-26T20:02:44.205699+0000","last_became_active":"2024-04-26T20:02:44.201948+0000","last_became_peered":"2024-04-26T20:02:44.201948+0000","last_unstale":"2024-04-26T20:02:44.205699+0000","last_undegraded":"2024-04-26T20:02:44.205699+0000","last_fullsized":"2024-04-26T20:02:44.205699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.031320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.218705+0000","last_change":"2024-04-26T20:02:44.215048+0000","last_active":"2024-04-26T20:02:44.218705+0000","last_peered":"2024-04-26T20:02:44.218705+0000","last_clean":"2024-04-26T20:02:44.218705+0000","last_became_active":"2024-04-26T20:02:44.214743+0000","last_became_peered":"2024-04-26T20:02:44.214743+0000","last_unstale":"2024-04-26T20:02:44.218705+0000","last_undegraded":"2024-04-26T20:02:44.218705+0000","last_fullsized":"2024-04-26T20:02:44.218705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:26.109862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225672+0000","last_change":"2024-04-26T20:02:44.213192+0000","last_active":"2024-04-26T20:02:44.225672+0000","last_peered":"2024-04-26T20:02:44.225672+0000","last_clean":"2024-04-26T20:02:44.225672+0000","last_became_active":"2024-04-26T20:02:44.212831+0000","last_became_peered":"2024-04-26T20:02:44.212831+0000","last_unstale":"2024-04-26T20:02:44.225672+0000","last_undegraded":"2024-04-26T20:02:44.225672+0000","last_fullsized":"2024-04-26T20:02:44.225672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:12.438185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.210232+0000","last_change":"2024-04-26T20:02:44.207071+0000","last_active":"2024-04-26T20:02:44.210232+0000","last_peered":"2024-04-26T20:02:44.210232+0000","last_clean":"2024-04-26T20:02:44.210232+0000","last_became_active":"2024-04-26T20:02:44.206006+0000","last_became_peered":"2024-04-26T20:02:44.206006+0000","last_unstale":"2024-04-26T20:02:44.210232+0000","last_undegraded":"2024-04-26T20:02:44.210232+0000","last_fullsized":"2024-04-26T20:02:44.210232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:19.045556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223985+0000","last_change":"2024-04-26T20:02:44.211050+0000","last_active":"2024-04-26T20:02:44.223985+0000","last_peered":"2024-04-26T20:02:44.223985+0000","last_clean":"2024-04-26T20:02:44.223985+0000","last_became_active":"2024-04-26T20:02:44.210140+0000","last_became_peered":"2024-04-26T20:02:44.210140+0000","last_unstale":"2024-04-26T20:02:44.223985+0000","last_undegraded":"2024-04-26T20:02:44.223985+0000","last_fullsized":"2024-04-26T20:02:44.223985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:11.659616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235046+0000","last_change":"2024-04-26T20:02:44.232299+0000","last_active":"2024-04-26T20:02:44.235046+0000","last_peered":"2024-04-26T20:02:44.235046+0000","last_clean":"2024-04-26T20:02:44.235046+0000","last_became_active":"2024-04-26T20:02:44.230283+0000","last_became_peered":"2024-04-26T20:02:44.230283+0000","last_unstale":"2024-04-26T20:02:44.235046+0000","last_undegraded":"2024-04-26T20:02:44.235046+0000","last_fullsized":"2024-04-26T20:02:44.235046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:35.436478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235642+0000","last_change":"2024-04-26T20:02:44.231146+0000","last_active":"2024-04-26T20:02:44.235642+0000","last_peered":"2024-04-26T20:02:44.235642+0000","last_clean":"2024-04-26T20:02:44.235642+0000","last_became_active":"2024-04-26T20:02:44.230806+0000","last_became_peered":"2024-04-26T20:02:44.230806+0000","last_unstale":"2024-04-26T20:02:44.235642+0000","last_undegraded":"2024-04-26T20:02:44.235642+0000","last_fullsized":"2024-04-26T20:02:44.235642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:45.730488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237109+0000","last_change":"2024-04-26T20:02:44.233249+0000","last_active":"2024-04-26T20:02:44.237109+0000","last_peered":"2024-04-26T20:02:44.237109+0000","last_clean":"2024-04-26T20:02:44.237109+0000","last_became_active":"2024-04-26T20:02:44.229290+0000","last_became_peered":"2024-04-26T20:02:44.229290+0000","last_unstale":"2024-04-26T20:02:44.237109+0000","last_undegraded":"2024-04-26T20:02:44.237109+0000","last_fullsized":"2024-04-26T20:02:44.237109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:51.840715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.232877+0000","last_change":"2024-04-26T20:02:44.228379+0000","last_active":"2024-04-26T20:02:44.232877+0000","last_peered":"2024-04-26T20:02:44.232877+0000","last_clean":"2024-04-26T20:02:44.232877+0000","last_became_active":"2024-04-26T20:02:44.228061+0000","last_became_peered":"2024-04-26T20:02:44.228061+0000","last_unstale":"2024-04-26T20:02:44.232877+0000","last_undegraded":"2024-04-26T20:02:44.232877+0000","last_fullsized":"2024-04-26T20:02:44.232877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:45.730488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.217938+0000","last_change":"2024-04-26T20:02:44.211159+0000","last_active":"2024-04-26T20:02:44.217938+0000","last_peered":"2024-04-26T20:02:44.217938+0000","last_clean":"2024-04-26T20:02:44.217938+0000","last_became_active":"2024-04-26T20:02:44.210822+0000","last_became_peered":"2024-04-26T20:02:44.210822+0000","last_unstale":"2024-04-26T20:02:44.217938+0000","last_undegraded":"2024-04-26T20:02:44.217938+0000","last_fullsized":"2024-04-26T20:02:44.217938+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:58.616520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234623+0000","last_change":"2024-04-26T20:02:44.228548+0000","last_active":"2024-04-26T20:02:44.234623+0000","last_peered":"2024-04-26T20:02:44.234623+0000","last_clean":"2024-04-26T20:02:44.234623+0000","last_became_active":"2024-04-26T20:02:44.228270+0000","last_became_peered":"2024-04-26T20:02:44.228270+0000","last_unstale":"2024-04-26T20:02:44.234623+0000","last_undegraded":"2024-04-26T20:02:44.234623+0000","last_fullsized":"2024-04-26T20:02:44.234623+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.664805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.219713+0000","last_change":"2024-04-26T20:02:44.214877+0000","last_active":"2024-04-26T20:02:44.219713+0000","last_peered":"2024-04-26T20:02:44.219713+0000","last_clean":"2024-04-26T20:02:44.219713+0000","last_became_active":"2024-04-26T20:02:44.214509+0000","last_became_peered":"2024-04-26T20:02:44.214509+0000","last_unstale":"2024-04-26T20:02:44.219713+0000","last_undegraded":"2024-04-26T20:02:44.219713+0000","last_fullsized":"2024-04-26T20:02:44.219713+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:42.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228847+0000","last_change":"2024-04-26T20:02:44.220061+0000","last_active":"2024-04-26T20:02:44.228847+0000","last_peered":"2024-04-26T20:02:44.228847+0000","last_clean":"2024-04-26T20:02:44.228847+0000","last_became_active":"2024-04-26T20:02:44.219714+0000","last_became_peered":"2024-04-26T20:02:44.219714+0000","last_unstale":"2024-04-26T20:02:44.228847+0000","last_undegraded":"2024-04-26T20:02:44.228847+0000","last_fullsized":"2024-04-26T20:02:44.228847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:54.369312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235022+0000","last_change":"2024-04-26T20:02:44.232167+0000","last_active":"2024-04-26T20:02:44.235022+0000","last_peered":"2024-04-26T20:02:44.235022+0000","last_clean":"2024-04-26T20:02:44.235022+0000","last_became_active":"2024-04-26T20:02:44.230634+0000","last_became_peered":"2024-04-26T20:02:44.230634+0000","last_unstale":"2024-04-26T20:02:44.235022+0000","last_undegraded":"2024-04-26T20:02:44.235022+0000","last_fullsized":"2024-04-26T20:02:44.235022+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:35.852379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205363+0000","last_change":"2024-04-26T20:02:44.199914+0000","last_active":"2024-04-26T20:02:44.205363+0000","last_peered":"2024-04-26T20:02:44.205363+0000","last_clean":"2024-04-26T20:02:44.205363+0000","last_became_active":"2024-04-26T20:02:44.199049+0000","last_became_peered":"2024-04-26T20:02:44.199049+0000","last_unstale":"2024-04-26T20:02:44.205363+0000","last_undegraded":"2024-04-26T20:02:44.205363+0000","last_fullsized":"2024-04-26T20:02:44.205363+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:39.710138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226700+0000","last_change":"2024-04-26T20:02:44.218491+0000","last_active":"2024-04-26T20:02:44.226700+0000","last_peered":"2024-04-26T20:02:44.226700+0000","last_clean":"2024-04-26T20:02:44.226700+0000","last_became_active":"2024-04-26T20:02:44.213787+0000","last_became_peered":"2024-04-26T20:02:44.213787+0000","last_unstale":"2024-04-26T20:02:44.226700+0000","last_undegraded":"2024-04-26T20:02:44.226700+0000","last_fullsized":"2024-04-26T20:02:44.226700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:39.271238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.218935+0000","last_change":"2024-04-26T20:02:44.210984+0000","last_active":"2024-04-26T20:02:44.218935+0000","last_peered":"2024-04-26T20:02:44.218935+0000","last_clean":"2024-04-26T20:02:44.218935+0000","last_became_active":"2024-04-26T20:02:44.210665+0000","last_became_peered":"2024-04-26T20:02:44.210665+0000","last_unstale":"2024-04-26T20:02:44.218935+0000","last_undegraded":"2024-04-26T20:02:44.218935+0000","last_fullsized":"2024-04-26T20:02:44.218935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:47.950924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.221886+0000","last_change":"2024-04-26T20:02:44.218084+0000","last_active":"2024-04-26T20:02:44.221886+0000","last_peered":"2024-04-26T20:02:44.221886+0000","last_clean":"2024-04-26T20:02:44.221886+0000","last_became_active":"2024-04-26T20:02:44.217714+0000","last_became_peered":"2024-04-26T20:02:44.217714+0000","last_unstale":"2024-04-26T20:02:44.221886+0000","last_undegraded":"2024-04-26T20:02:44.221886+0000","last_fullsized":"2024-04-26T20:02:44.221886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:02.842485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235632+0000","last_change":"2024-04-26T20:02:44.228336+0000","last_active":"2024-04-26T20:02:44.235632+0000","last_peered":"2024-04-26T20:02:44.235632+0000","last_clean":"2024-04-26T20:02:44.235632+0000","last_became_active":"2024-04-26T20:02:44.228031+0000","last_became_peered":"2024-04-26T20:02:44.228031+0000","last_unstale":"2024-04-26T20:02:44.235632+0000","last_undegraded":"2024-04-26T20:02:44.235632+0000","last_fullsized":"2024-04-26T20:02:44.235632+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:52:06.221711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228863+0000","last_change":"2024-04-26T20:02:44.218940+0000","last_active":"2024-04-26T20:02:44.228863+0000","last_peered":"2024-04-26T20:02:44.228863+0000","last_clean":"2024-04-26T20:02:44.228863+0000","last_became_active":"2024-04-26T20:02:44.218673+0000","last_became_peered":"2024-04-26T20:02:44.218673+0000","last_unstale":"2024-04-26T20:02:44.228863+0000","last_undegraded":"2024-04-26T20:02:44.228863+0000","last_fullsized":"2024-04-26T20:02:44.228863+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:15.492197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.239290+0000","last_change":"2024-04-26T20:02:44.236625+0000","last_active":"2024-04-26T20:02:44.239290+0000","last_peered":"2024-04-26T20:02:44.239290+0000","last_clean":"2024-04-26T20:02:44.239290+0000","last_became_active":"2024-04-26T20:02:44.233643+0000","last_became_peered":"2024-04-26T20:02:44.233643+0000","last_unstale":"2024-04-26T20:02:44.239290+0000","last_undegraded":"2024-04-26T20:02:44.239290+0000","last_fullsized":"2024-04-26T20:02:44.239290+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:17.838609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234915+0000","last_change":"2024-04-26T20:02:44.227643+0000","last_active":"2024-04-26T20:02:44.234915+0000","last_peered":"2024-04-26T20:02:44.234915+0000","last_clean":"2024-04-26T20:02:44.234915+0000","last_became_active":"2024-04-26T20:02:44.227376+0000","last_became_peered":"2024-04-26T20:02:44.227376+0000","last_unstale":"2024-04-26T20:02:44.234915+0000","last_undegraded":"2024-04-26T20:02:44.234915+0000","last_fullsized":"2024-04-26T20:02:44.234915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:39.710138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228451+0000","last_change":"2024-04-26T20:02:44.219382+0000","last_active":"2024-04-26T20:02:44.228451+0000","last_peered":"2024-04-26T20:02:44.228451+0000","last_clean":"2024-04-26T20:02:44.228451+0000","last_became_active":"2024-04-26T20:02:44.219064+0000","last_became_peered":"2024-04-26T20:02:44.219064+0000","last_unstale":"2024-04-26T20:02:44.228451+0000","last_undegraded":"2024-04-26T20:02:44.228451+0000","last_fullsized":"2024-04-26T20:02:44.228451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:50.846924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.231406+0000","last_change":"2024-04-26T20:02:44.227502+0000","last_active":"2024-04-26T20:02:44.231406+0000","last_peered":"2024-04-26T20:02:44.231406+0000","last_clean":"2024-04-26T20:02:44.231406+0000","last_became_active":"2024-04-26T20:02:44.227208+0000","last_became_peered":"2024-04-26T20:02:44.227208+0000","last_unstale":"2024-04-26T20:02:44.231406+0000","last_undegraded":"2024-04-26T20:02:44.231406+0000","last_fullsized":"2024-04-26T20:02:44.231406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:39.824363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235124+0000","last_change":"2024-04-26T20:02:44.228350+0000","last_active":"2024-04-26T20:02:44.235124+0000","last_peered":"2024-04-26T20:02:44.235124+0000","last_clean":"2024-04-26T20:02:44.235124+0000","last_became_active":"2024-04-26T20:02:44.228018+0000","last_became_peered":"2024-04-26T20:02:44.228018+0000","last_unstale":"2024-04-26T20:02:44.235124+0000","last_undegraded":"2024-04-26T20:02:44.235124+0000","last_fullsized":"2024-04-26T20:02:44.235124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:53:34.293453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223057+0000","last_change":"2024-04-26T20:02:44.210322+0000","last_active":"2024-04-26T20:02:44.223057+0000","last_peered":"2024-04-26T20:02:44.223057+0000","last_clean":"2024-04-26T20:02:44.223057+0000","last_became_active":"2024-04-26T20:02:44.207157+0000","last_became_peered":"2024-04-26T20:02:44.207157+0000","last_unstale":"2024-04-26T20:02:44.223057+0000","last_undegraded":"2024-04-26T20:02:44.223057+0000","last_fullsized":"2024-04-26T20:02:44.223057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:28.540865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228886+0000","last_change":"2024-04-26T20:02:44.219921+0000","last_active":"2024-04-26T20:02:44.228886+0000","last_peered":"2024-04-26T20:02:44.228886+0000","last_clean":"2024-04-26T20:02:44.228886+0000","last_became_active":"2024-04-26T20:02:44.219646+0000","last_became_peered":"2024-04-26T20:02:44.219646+0000","last_unstale":"2024-04-26T20:02:44.228886+0000","last_undegraded":"2024-04-26T20:02:44.228886+0000","last_fullsized":"2024-04-26T20:02:44.228886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:52.177288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229006+0000","last_change":"2024-04-26T20:02:44.220753+0000","last_active":"2024-04-26T20:02:44.229006+0000","last_peered":"2024-04-26T20:02:44.229006+0000","last_clean":"2024-04-26T20:02:44.229006+0000","last_became_active":"2024-04-26T20:02:44.220101+0000","last_became_peered":"2024-04-26T20:02:44.220101+0000","last_unstale":"2024-04-26T20:02:44.229006+0000","last_undegraded":"2024-04-26T20:02:44.229006+0000","last_fullsized":"2024-04-26T20:02:44.229006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.650660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223943+0000","last_change":"2024-04-26T20:02:44.209633+0000","last_active":"2024-04-26T20:02:44.223943+0000","last_peered":"2024-04-26T20:02:44.223943+0000","last_clean":"2024-04-26T20:02:44.223943+0000","last_became_active":"2024-04-26T20:02:44.202200+0000","last_became_peered":"2024-04-26T20:02:44.202200+0000","last_unstale":"2024-04-26T20:02:44.223943+0000","last_undegraded":"2024-04-26T20:02:44.223943+0000","last_fullsized":"2024-04-26T20:02:44.223943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:14.948053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237073+0000","last_change":"2024-04-26T20:02:44.231591+0000","last_active":"2024-04-26T20:02:44.237073+0000","last_peered":"2024-04-26T20:02:44.237073+0000","last_clean":"2024-04-26T20:02:44.237073+0000","last_became_active":"2024-04-26T20:02:44.229473+0000","last_became_peered":"2024-04-26T20:02:44.229473+0000","last_unstale":"2024-04-26T20:02:44.237073+0000","last_undegraded":"2024-04-26T20:02:44.237073+0000","last_fullsized":"2024-04-26T20:02:44.237073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:42.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.247242+0000","last_change":"2024-04-26T20:02:43.202527+0000","last_active":"2024-04-26T20:02:44.247242+0000","last_peered":"2024-04-26T20:02:44.247242+0000","last_clean":"2024-04-26T20:02:44.247242+0000","last_became_active":"2024-04-26T20:02:43.202216+0000","last_became_peered":"2024-04-26T20:02:43.202216+0000","last_unstale":"2024-04-26T20:02:44.247242+0000","last_undegraded":"2024-04-26T20:02:44.247242+0000","last_fullsized":"2024-04-26T20:02:44.247242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:29.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237061+0000","last_change":"2024-04-26T20:02:44.232411+0000","last_active":"2024-04-26T20:02:44.237061+0000","last_peered":"2024-04-26T20:02:44.237061+0000","last_clean":"2024-04-26T20:02:44.237061+0000","last_became_active":"2024-04-26T20:02:44.229769+0000","last_became_peered":"2024-04-26T20:02:44.229769+0000","last_unstale":"2024-04-26T20:02:44.237061+0000","last_undegraded":"2024-04-26T20:02:44.237061+0000","last_fullsized":"2024-04-26T20:02:44.237061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:58.358025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234723+0000","last_change":"2024-04-26T20:02:43.179684+0000","last_active":"2024-04-26T20:02:44.234723+0000","last_peered":"2024-04-26T20:02:44.234723+0000","last_clean":"2024-04-26T20:02:44.234723+0000","last_became_active":"2024-04-26T20:02:43.179299+0000","last_became_peered":"2024-04-26T20:02:43.179299+0000","last_unstale":"2024-04-26T20:02:44.234723+0000","last_undegraded":"2024-04-26T20:02:44.234723+0000","last_fullsized":"2024-04-26T20:02:44.234723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:14.478180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235760+0000","last_change":"2024-04-26T20:02:44.221688+0000","last_active":"2024-04-26T20:02:44.235760+0000","last_peered":"2024-04-26T20:02:44.235760+0000","last_clean":"2024-04-26T20:02:44.235760+0000","last_became_active":"2024-04-26T20:02:44.221367+0000","last_became_peered":"2024-04-26T20:02:44.221367+0000","last_unstale":"2024-04-26T20:02:44.235760+0000","last_undegraded":"2024-04-26T20:02:44.235760+0000","last_fullsized":"2024-04-26T20:02:44.235760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:47.708744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236502+0000","last_change":"2024-04-26T20:02:43.202054+0000","last_active":"2024-04-26T20:02:44.236502+0000","last_peered":"2024-04-26T20:02:44.236502+0000","last_clean":"2024-04-26T20:02:44.236502+0000","last_became_active":"2024-04-26T20:02:43.201768+0000","last_became_peered":"2024-04-26T20:02:43.201768+0000","last_unstale":"2024-04-26T20:02:44.236502+0000","last_undegraded":"2024-04-26T20:02:44.236502+0000","last_fullsized":"2024-04-26T20:02:44.236502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:46.694726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235511+0000","last_change":"2024-04-26T20:02:44.225032+0000","last_active":"2024-04-26T20:02:44.235511+0000","last_peered":"2024-04-26T20:02:44.235511+0000","last_clean":"2024-04-26T20:02:44.235511+0000","last_became_active":"2024-04-26T20:02:44.224669+0000","last_became_peered":"2024-04-26T20:02:44.224669+0000","last_unstale":"2024-04-26T20:02:44.235511+0000","last_undegraded":"2024-04-26T20:02:44.235511+0000","last_fullsized":"2024-04-26T20:02:44.235511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:47.950924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237580+0000","last_change":"2024-04-26T20:02:43.190355+0000","last_active":"2024-04-26T20:02:44.237580+0000","last_peered":"2024-04-26T20:02:44.237580+0000","last_clean":"2024-04-26T20:02:44.237580+0000","last_became_active":"2024-04-26T20:02:43.190059+0000","last_became_peered":"2024-04-26T20:02:43.190059+0000","last_unstale":"2024-04-26T20:02:44.237580+0000","last_undegraded":"2024-04-26T20:02:44.237580+0000","last_fullsized":"2024-04-26T20:02:44.237580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:38.176561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.207961+0000","last_change":"2024-04-26T20:02:44.204413+0000","last_active":"2024-04-26T20:02:44.207961+0000","last_peered":"2024-04-26T20:02:44.207961+0000","last_clean":"2024-04-26T20:02:44.207961+0000","last_became_active":"2024-04-26T20:02:44.203972+0000","last_became_peered":"2024-04-26T20:02:44.203972+0000","last_unstale":"2024-04-26T20:02:44.207961+0000","last_undegraded":"2024-04-26T20:02:44.207961+0000","last_fullsized":"2024-04-26T20:02:44.207961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:25.653238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226896+0000","last_change":"2024-04-26T20:02:43.192634+0000","last_active":"2024-04-26T20:02:44.226896+0000","last_peered":"2024-04-26T20:02:44.226896+0000","last_clean":"2024-04-26T20:02:44.226896+0000","last_became_active":"2024-04-26T20:02:43.192306+0000","last_became_peered":"2024-04-26T20:02:43.192306+0000","last_unstale":"2024-04-26T20:02:44.226896+0000","last_undegraded":"2024-04-26T20:02:44.226896+0000","last_fullsized":"2024-04-26T20:02:44.226896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:36.450019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235161+0000","last_change":"2024-04-26T20:02:44.228277+0000","last_active":"2024-04-26T20:02:44.235161+0000","last_peered":"2024-04-26T20:02:44.235161+0000","last_clean":"2024-04-26T20:02:44.235161+0000","last_became_active":"2024-04-26T20:02:44.227905+0000","last_became_peered":"2024-04-26T20:02:44.227905+0000","last_unstale":"2024-04-26T20:02:44.235161+0000","last_undegraded":"2024-04-26T20:02:44.235161+0000","last_fullsized":"2024-04-26T20:02:44.235161+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.031320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235939+0000","last_change":"2024-04-26T20:02:43.192717+0000","last_active":"2024-04-26T20:02:44.235939+0000","last_peered":"2024-04-26T20:02:44.235939+0000","last_clean":"2024-04-26T20:02:44.235939+0000","last_became_active":"2024-04-26T20:02:43.192413+0000","last_became_peered":"2024-04-26T20:02:43.192413+0000","last_unstale":"2024-04-26T20:02:44.235939+0000","last_undegraded":"2024-04-26T20:02:44.235939+0000","last_fullsized":"2024-04-26T20:02:44.235939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:04.201771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.220445+0000","last_change":"2024-04-26T20:02:44.217953+0000","last_active":"2024-04-26T20:02:44.220445+0000","last_peered":"2024-04-26T20:02:44.220445+0000","last_clean":"2024-04-26T20:02:44.220445+0000","last_became_active":"2024-04-26T20:02:44.217548+0000","last_became_peered":"2024-04-26T20:02:44.217548+0000","last_unstale":"2024-04-26T20:02:44.220445+0000","last_undegraded":"2024-04-26T20:02:44.220445+0000","last_fullsized":"2024-04-26T20:02:44.220445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:58.286972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.183862+0000","last_change":"2024-04-26T20:02:43.194019+0000","last_active":"2024-04-26T20:02:44.183862+0000","last_peered":"2024-04-26T20:02:44.183862+0000","last_clean":"2024-04-26T20:02:44.183862+0000","last_became_active":"2024-04-26T20:02:43.193689+0000","last_became_peered":"2024-04-26T20:02:43.193689+0000","last_unstale":"2024-04-26T20:02:44.183862+0000","last_undegraded":"2024-04-26T20:02:44.183862+0000","last_fullsized":"2024-04-26T20:02:44.183862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:05.375211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.219691+0000","last_change":"2024-04-26T20:02:44.213286+0000","last_active":"2024-04-26T20:02:44.219691+0000","last_peered":"2024-04-26T20:02:44.219691+0000","last_clean":"2024-04-26T20:02:44.219691+0000","last_became_active":"2024-04-26T20:02:44.212956+0000","last_became_peered":"2024-04-26T20:02:44.212956+0000","last_unstale":"2024-04-26T20:02:44.219691+0000","last_undegraded":"2024-04-26T20:02:44.219691+0000","last_fullsized":"2024-04-26T20:02:44.219691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:44.777171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.204156+0000","last_change":"2024-04-26T20:02:43.179363+0000","last_active":"2024-04-26T20:02:44.204156+0000","last_peered":"2024-04-26T20:02:44.204156+0000","last_clean":"2024-04-26T20:02:44.204156+0000","last_became_active":"2024-04-26T20:02:43.179053+0000","last_became_peered":"2024-04-26T20:02:43.179053+0000","last_unstale":"2024-04-26T20:02:44.204156+0000","last_undegraded":"2024-04-26T20:02:44.204156+0000","last_fullsized":"2024-04-26T20:02:44.204156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:53.470531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228424+0000","last_change":"2024-04-26T20:02:44.216999+0000","last_active":"2024-04-26T20:02:44.228424+0000","last_peered":"2024-04-26T20:02:44.228424+0000","last_clean":"2024-04-26T20:02:44.228424+0000","last_became_active":"2024-04-26T20:02:44.216694+0000","last_became_peered":"2024-04-26T20:02:44.216694+0000","last_unstale":"2024-04-26T20:02:44.228424+0000","last_undegraded":"2024-04-26T20:02:44.228424+0000","last_fullsized":"2024-04-26T20:02:44.228424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:56.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.212251+0000","last_change":"2024-04-26T20:02:43.188207+0000","last_active":"2024-04-26T20:02:44.212251+0000","last_peered":"2024-04-26T20:02:44.212251+0000","last_clean":"2024-04-26T20:02:44.212251+0000","last_became_active":"2024-04-26T20:02:43.187889+0000","last_became_peered":"2024-04-26T20:02:43.187889+0000","last_unstale":"2024-04-26T20:02:44.212251+0000","last_undegraded":"2024-04-26T20:02:44.212251+0000","last_fullsized":"2024-04-26T20:02:44.212251+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:05.375211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242234+0000","last_change":"2024-04-26T20:02:44.236248+0000","last_active":"2024-04-26T20:02:44.242234+0000","last_peered":"2024-04-26T20:02:44.242234+0000","last_clean":"2024-04-26T20:02:44.242234+0000","last_became_active":"2024-04-26T20:02:44.235020+0000","last_became_peered":"2024-04-26T20:02:44.235020+0000","last_unstale":"2024-04-26T20:02:44.242234+0000","last_undegraded":"2024-04-26T20:02:44.242234+0000","last_fullsized":"2024-04-26T20:02:44.242234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:37.193478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.196912+0000","last_change":"2024-04-26T20:02:43.198670+0000","last_active":"2024-04-26T20:02:44.196912+0000","last_peered":"2024-04-26T20:02:44.196912+0000","last_clean":"2024-04-26T20:02:44.196912+0000","last_became_active":"2024-04-26T20:02:43.197285+0000","last_became_peered":"2024-04-26T20:02:43.197285+0000","last_unstale":"2024-04-26T20:02:44.196912+0000","last_undegraded":"2024-04-26T20:02:44.196912+0000","last_fullsized":"2024-04-26T20:02:44.196912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:40.535868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224360+0000","last_change":"2024-04-26T20:02:44.214898+0000","last_active":"2024-04-26T20:02:44.224360+0000","last_peered":"2024-04-26T20:02:44.224360+0000","last_clean":"2024-04-26T20:02:44.224360+0000","last_became_active":"2024-04-26T20:02:44.214586+0000","last_became_peered":"2024-04-26T20:02:44.214586+0000","last_unstale":"2024-04-26T20:02:44.224360+0000","last_undegraded":"2024-04-26T20:02:44.224360+0000","last_fullsized":"2024-04-26T20:02:44.224360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:37.848234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234386+0000","last_change":"2024-04-26T20:02:43.192534+0000","last_active":"2024-04-26T20:02:44.234386+0000","last_peered":"2024-04-26T20:02:44.234386+0000","last_clean":"2024-04-26T20:02:44.234386+0000","last_became_active":"2024-04-26T20:02:43.191930+0000","last_became_peered":"2024-04-26T20:02:43.191930+0000","last_unstale":"2024-04-26T20:02:44.234386+0000","last_undegraded":"2024-04-26T20:02:44.234386+0000","last_fullsized":"2024-04-26T20:02:44.234386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:38.535802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229216+0000","last_change":"2024-04-26T20:02:44.224559+0000","last_active":"2024-04-26T20:02:44.229216+0000","last_peered":"2024-04-26T20:02:44.229216+0000","last_clean":"2024-04-26T20:02:44.229216+0000","last_became_active":"2024-04-26T20:02:44.221905+0000","last_became_peered":"2024-04-26T20:02:44.221905+0000","last_unstale":"2024-04-26T20:02:44.229216+0000","last_undegraded":"2024-04-26T20:02:44.229216+0000","last_fullsized":"2024-04-26T20:02:44.229216+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:33.342047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.239806+0000","last_change":"2024-04-26T20:02:43.179957+0000","last_active":"2024-04-26T20:02:44.239806+0000","last_peered":"2024-04-26T20:02:44.239806+0000","last_clean":"2024-04-26T20:02:44.239806+0000","last_became_active":"2024-04-26T20:02:43.179606+0000","last_became_peered":"2024-04-26T20:02:43.179606+0000","last_unstale":"2024-04-26T20:02:44.239806+0000","last_undegraded":"2024-04-26T20:02:44.239806+0000","last_fullsized":"2024-04-26T20:02:44.239806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:53.355295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236309+0000","last_change":"2024-04-26T20:02:44.225784+0000","last_active":"2024-04-26T20:02:44.236309+0000","last_peered":"2024-04-26T20:02:44.236309+0000","last_clean":"2024-04-26T20:02:44.236309+0000","last_became_active":"2024-04-26T20:02:44.225496+0000","last_became_peered":"2024-04-26T20:02:44.225496+0000","last_unstale":"2024-04-26T20:02:44.236309+0000","last_undegraded":"2024-04-26T20:02:44.236309+0000","last_fullsized":"2024-04-26T20:02:44.236309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:58.616520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237374+0000","last_change":"2024-04-26T20:02:43.192741+0000","last_active":"2024-04-26T20:02:44.237374+0000","last_peered":"2024-04-26T20:02:44.237374+0000","last_clean":"2024-04-26T20:02:44.237374+0000","last_became_active":"2024-04-26T20:02:43.192405+0000","last_became_peered":"2024-04-26T20:02:43.192405+0000","last_unstale":"2024-04-26T20:02:44.237374+0000","last_undegraded":"2024-04-26T20:02:44.237374+0000","last_fullsized":"2024-04-26T20:02:44.237374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:01.828467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229365+0000","last_change":"2024-04-26T20:02:44.217064+0000","last_active":"2024-04-26T20:02:44.229365+0000","last_peered":"2024-04-26T20:02:44.229365+0000","last_clean":"2024-04-26T20:02:44.229365+0000","last_became_active":"2024-04-26T20:02:44.216719+0000","last_became_peered":"2024-04-26T20:02:44.216719+0000","last_unstale":"2024-04-26T20:02:44.229365+0000","last_undegraded":"2024-04-26T20:02:44.229365+0000","last_fullsized":"2024-04-26T20:02:44.229365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:59.254006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191403+0000","last_change":"2024-04-26T20:02:43.198415+0000","last_active":"2024-04-26T20:02:44.191403+0000","last_peered":"2024-04-26T20:02:44.191403+0000","last_clean":"2024-04-26T20:02:44.191403+0000","last_became_active":"2024-04-26T20:02:43.198143+0000","last_became_peered":"2024-04-26T20:02:43.198143+0000","last_unstale":"2024-04-26T20:02:44.191403+0000","last_undegraded":"2024-04-26T20:02:44.191403+0000","last_fullsized":"2024-04-26T20:02:44.191403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:12.625877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224970+0000","last_change":"2024-04-26T20:02:44.222241+0000","last_active":"2024-04-26T20:02:44.224970+0000","last_peered":"2024-04-26T20:02:44.224970+0000","last_clean":"2024-04-26T20:02:44.224970+0000","last_became_active":"2024-04-26T20:02:44.221094+0000","last_became_peered":"2024-04-26T20:02:44.221094+0000","last_unstale":"2024-04-26T20:02:44.224970+0000","last_undegraded":"2024-04-26T20:02:44.224970+0000","last_fullsized":"2024-04-26T20:02:44.224970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:51.840715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191896+0000","last_change":"2024-04-26T20:02:43.196404+0000","last_active":"2024-04-26T20:02:44.191896+0000","last_peered":"2024-04-26T20:02:44.191896+0000","last_clean":"2024-04-26T20:02:44.191896+0000","last_became_active":"2024-04-26T20:02:43.196108+0000","last_became_peered":"2024-04-26T20:02:43.196108+0000","last_unstale":"2024-04-26T20:02:44.191896+0000","last_undegraded":"2024-04-26T20:02:44.191896+0000","last_fullsized":"2024-04-26T20:02:44.191896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:20.412508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.231471+0000","last_change":"2024-04-26T20:02:44.224192+0000","last_active":"2024-04-26T20:02:44.231471+0000","last_peered":"2024-04-26T20:02:44.231471+0000","last_clean":"2024-04-26T20:02:44.231471+0000","last_became_active":"2024-04-26T20:02:44.220837+0000","last_became_peered":"2024-04-26T20:02:44.220837+0000","last_unstale":"2024-04-26T20:02:44.231471+0000","last_undegraded":"2024-04-26T20:02:44.231471+0000","last_fullsized":"2024-04-26T20:02:44.231471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:00.635462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.243044+0000","last_change":"2024-04-26T20:02:43.197576+0000","last_active":"2024-04-26T20:02:44.243044+0000","last_peered":"2024-04-26T20:02:44.243044+0000","last_clean":"2024-04-26T20:02:44.243044+0000","last_became_active":"2024-04-26T20:02:43.197290+0000","last_became_peered":"2024-04-26T20:02:43.197290+0000","last_unstale":"2024-04-26T20:02:44.243044+0000","last_undegraded":"2024-04-26T20:02:44.243044+0000","last_fullsized":"2024-04-26T20:02:44.243044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:35.363409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.217403+0000","last_change":"2024-04-26T20:02:44.215169+0000","last_active":"2024-04-26T20:02:44.217403+0000","last_peered":"2024-04-26T20:02:44.217403+0000","last_clean":"2024-04-26T20:02:44.217403+0000","last_became_active":"2024-04-26T20:02:44.214836+0000","last_became_peered":"2024-04-26T20:02:44.214836+0000","last_unstale":"2024-04-26T20:02:44.217403+0000","last_undegraded":"2024-04-26T20:02:44.217403+0000","last_fullsized":"2024-04-26T20:02:44.217403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:22.227868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238515+0000","last_change":"2024-04-26T20:02:43.199855+0000","last_active":"2024-04-26T20:02:44.238515+0000","last_peered":"2024-04-26T20:02:44.238515+0000","last_clean":"2024-04-26T20:02:44.238515+0000","last_became_active":"2024-04-26T20:02:43.194522+0000","last_became_peered":"2024-04-26T20:02:43.194522+0000","last_unstale":"2024-04-26T20:02:44.238515+0000","last_undegraded":"2024-04-26T20:02:44.238515+0000","last_fullsized":"2024-04-26T20:02:44.238515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:20.072021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242210+0000","last_change":"2024-04-26T20:02:44.235859+0000","last_active":"2024-04-26T20:02:44.242210+0000","last_peered":"2024-04-26T20:02:44.242210+0000","last_clean":"2024-04-26T20:02:44.242210+0000","last_became_active":"2024-04-26T20:02:44.232674+0000","last_became_peered":"2024-04-26T20:02:44.232674+0000","last_unstale":"2024-04-26T20:02:44.242210+0000","last_undegraded":"2024-04-26T20:02:44.242210+0000","last_fullsized":"2024-04-26T20:02:44.242210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:56.396247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236809+0000","last_change":"2024-04-26T20:02:43.201318+0000","last_active":"2024-04-26T20:02:44.236809+0000","last_peered":"2024-04-26T20:02:44.236809+0000","last_clean":"2024-04-26T20:02:44.236809+0000","last_became_active":"2024-04-26T20:02:43.201012+0000","last_became_peered":"2024-04-26T20:02:43.201012+0000","last_unstale":"2024-04-26T20:02:44.236809+0000","last_undegraded":"2024-04-26T20:02:44.236809+0000","last_fullsized":"2024-04-26T20:02:44.236809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:11.319483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225106+0000","last_change":"2024-04-26T20:02:44.218274+0000","last_active":"2024-04-26T20:02:44.225106+0000","last_peered":"2024-04-26T20:02:44.225106+0000","last_clean":"2024-04-26T20:02:44.225106+0000","last_became_active":"2024-04-26T20:02:44.218018+0000","last_became_peered":"2024-04-26T20:02:44.218018+0000","last_unstale":"2024-04-26T20:02:44.225106+0000","last_undegraded":"2024-04-26T20:02:44.225106+0000","last_fullsized":"2024-04-26T20:02:44.225106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:52:06.221711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237711+0000","last_change":"2024-04-26T20:02:43.196074+0000","last_active":"2024-04-26T20:02:44.237711+0000","last_peered":"2024-04-26T20:02:44.237711+0000","last_clean":"2024-04-26T20:02:44.237711+0000","last_became_active":"2024-04-26T20:02:43.195148+0000","last_became_peered":"2024-04-26T20:02:43.195148+0000","last_unstale":"2024-04-26T20:02:44.237711+0000","last_undegraded":"2024-04-26T20:02:44.237711+0000","last_fullsized":"2024-04-26T20:02:44.237711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:13.940582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225410+0000","last_change":"2024-04-26T20:02:44.221039+0000","last_active":"2024-04-26T20:02:44.225410+0000","last_peered":"2024-04-26T20:02:44.225410+0000","last_clean":"2024-04-26T20:02:44.225410+0000","last_became_active":"2024-04-26T20:02:44.220749+0000","last_became_peered":"2024-04-26T20:02:44.220749+0000","last_unstale":"2024-04-26T20:02:44.225410+0000","last_undegraded":"2024-04-26T20:02:44.225410+0000","last_fullsized":"2024-04-26T20:02:44.225410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.133315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.209498+0000","last_change":"2024-04-26T20:02:43.201376+0000","last_active":"2024-04-26T20:02:44.209498+0000","last_peered":"2024-04-26T20:02:44.209498+0000","last_clean":"2024-04-26T20:02:44.209498+0000","last_became_active":"2024-04-26T20:02:43.201092+0000","last_became_peered":"2024-04-26T20:02:43.201092+0000","last_unstale":"2024-04-26T20:02:44.209498+0000","last_undegraded":"2024-04-26T20:02:44.209498+0000","last_fullsized":"2024-04-26T20:02:44.209498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:15.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223084+0000","last_change":"2024-04-26T20:02:44.211768+0000","last_active":"2024-04-26T20:02:44.223084+0000","last_peered":"2024-04-26T20:02:44.223084+0000","last_clean":"2024-04-26T20:02:44.223084+0000","last_became_active":"2024-04-26T20:02:44.211342+0000","last_became_peered":"2024-04-26T20:02:44.211342+0000","last_unstale":"2024-04-26T20:02:44.223084+0000","last_undegraded":"2024-04-26T20:02:44.223084+0000","last_fullsized":"2024-04-26T20:02:44.223084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:22.973327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.190017+0000","last_change":"2024-04-26T20:02:43.191853+0000","last_active":"2024-04-26T20:02:44.190017+0000","last_peered":"2024-04-26T20:02:44.190017+0000","last_clean":"2024-04-26T20:02:44.190017+0000","last_became_active":"2024-04-26T20:02:43.191514+0000","last_became_peered":"2024-04-26T20:02:43.191514+0000","last_unstale":"2024-04-26T20:02:44.190017+0000","last_undegraded":"2024-04-26T20:02:44.190017+0000","last_fullsized":"2024-04-26T20:02:44.190017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:42.640406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224848+0000","last_change":"2024-04-26T20:02:44.216878+0000","last_active":"2024-04-26T20:02:44.224848+0000","last_peered":"2024-04-26T20:02:44.224848+0000","last_clean":"2024-04-26T20:02:44.224848+0000","last_became_active":"2024-04-26T20:02:44.216564+0000","last_became_peered":"2024-04-26T20:02:44.216564+0000","last_unstale":"2024-04-26T20:02:44.224848+0000","last_undegraded":"2024-04-26T20:02:44.224848+0000","last_fullsized":"2024-04-26T20:02:44.224848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:05.473333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.192237+0000","last_change":"2024-04-26T20:02:43.201147+0000","last_active":"2024-04-26T20:02:44.192237+0000","last_peered":"2024-04-26T20:02:44.192237+0000","last_clean":"2024-04-26T20:02:44.192237+0000","last_became_active":"2024-04-26T20:02:43.200819+0000","last_became_peered":"2024-04-26T20:02:43.200819+0000","last_unstale":"2024-04-26T20:02:44.192237+0000","last_undegraded":"2024-04-26T20:02:44.192237+0000","last_fullsized":"2024-04-26T20:02:44.192237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:17.530347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223118+0000","last_change":"2024-04-26T20:02:44.213364+0000","last_active":"2024-04-26T20:02:44.223118+0000","last_peered":"2024-04-26T20:02:44.223118+0000","last_clean":"2024-04-26T20:02:44.223118+0000","last_became_active":"2024-04-26T20:02:44.212503+0000","last_became_peered":"2024-04-26T20:02:44.212503+0000","last_unstale":"2024-04-26T20:02:44.223118+0000","last_undegraded":"2024-04-26T20:02:44.223118+0000","last_fullsized":"2024-04-26T20:02:44.223118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:15.737350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.201317+0000","last_change":"2024-04-26T20:02:43.179908+0000","last_active":"2024-04-26T20:02:44.201317+0000","last_peered":"2024-04-26T20:02:44.201317+0000","last_clean":"2024-04-26T20:02:44.201317+0000","last_became_active":"2024-04-26T20:02:43.179279+0000","last_became_peered":"2024-04-26T20:02:43.179279+0000","last_unstale":"2024-04-26T20:02:44.201317+0000","last_undegraded":"2024-04-26T20:02:44.201317+0000","last_fullsized":"2024-04-26T20:02:44.201317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:15.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.232271+0000","last_change":"2024-04-26T20:02:44.228108+0000","last_active":"2024-04-26T20:02:44.232271+0000","last_peered":"2024-04-26T20:02:44.232271+0000","last_clean":"2024-04-26T20:02:44.232271+0000","last_became_active":"2024-04-26T20:02:44.227819+0000","last_became_peered":"2024-04-26T20:02:44.227819+0000","last_unstale":"2024-04-26T20:02:44.232271+0000","last_undegraded":"2024-04-26T20:02:44.232271+0000","last_fullsized":"2024-04-26T20:02:44.232271+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:13.957460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238317+0000","last_change":"2024-04-26T20:02:43.197861+0000","last_active":"2024-04-26T20:02:44.238317+0000","last_peered":"2024-04-26T20:02:44.238317+0000","last_clean":"2024-04-26T20:02:44.238317+0000","last_became_active":"2024-04-26T20:02:43.197576+0000","last_became_peered":"2024-04-26T20:02:43.197576+0000","last_unstale":"2024-04-26T20:02:44.238317+0000","last_undegraded":"2024-04-26T20:02:44.238317+0000","last_fullsized":"2024-04-26T20:02:44.238317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:07.448332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.207718+0000","last_change":"2024-04-26T20:02:44.206106+0000","last_active":"2024-04-26T20:02:44.207718+0000","last_peered":"2024-04-26T20:02:44.207718+0000","last_clean":"2024-04-26T20:02:44.207718+0000","last_became_active":"2024-04-26T20:02:44.205794+0000","last_became_peered":"2024-04-26T20:02:44.205794+0000","last_unstale":"2024-04-26T20:02:44.207718+0000","last_undegraded":"2024-04-26T20:02:44.207718+0000","last_fullsized":"2024-04-26T20:02:44.207718+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:43.489102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191422+0000","last_change":"2024-04-26T20:02:43.193051+0000","last_active":"2024-04-26T20:02:44.191422+0000","last_peered":"2024-04-26T20:02:44.191422+0000","last_clean":"2024-04-26T20:02:44.191422+0000","last_became_active":"2024-04-26T20:02:43.192761+0000","last_became_peered":"2024-04-26T20:02:43.192761+0000","last_unstale":"2024-04-26T20:02:44.191422+0000","last_undegraded":"2024-04-26T20:02:44.191422+0000","last_fullsized":"2024-04-26T20:02:44.191422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:25.562701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237150+0000","last_change":"2024-04-26T20:02:44.231343+0000","last_active":"2024-04-26T20:02:44.237150+0000","last_peered":"2024-04-26T20:02:44.237150+0000","last_clean":"2024-04-26T20:02:44.237150+0000","last_became_active":"2024-04-26T20:02:44.229669+0000","last_became_peered":"2024-04-26T20:02:44.229669+0000","last_unstale":"2024-04-26T20:02:44.237150+0000","last_undegraded":"2024-04-26T20:02:44.237150+0000","last_fullsized":"2024-04-26T20:02:44.237150+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:44.777171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.209872+0000","last_change":"2024-04-26T20:02:43.182063+0000","last_active":"2024-04-26T20:02:44.209872+0000","last_peered":"2024-04-26T20:02:44.209872+0000","last_clean":"2024-04-26T20:02:44.209872+0000","last_became_active":"2024-04-26T20:02:43.181661+0000","last_became_peered":"2024-04-26T20:02:43.181661+0000","last_unstale":"2024-04-26T20:02:44.209872+0000","last_undegraded":"2024-04-26T20:02:44.209872+0000","last_fullsized":"2024-04-26T20:02:44.209872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.194942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238882+0000","last_change":"2024-04-26T20:02:44.234787+0000","last_active":"2024-04-26T20:02:44.238882+0000","last_peered":"2024-04-26T20:02:44.238882+0000","last_clean":"2024-04-26T20:02:44.238882+0000","last_became_active":"2024-04-26T20:02:44.233632+0000","last_became_peered":"2024-04-26T20:02:44.233632+0000","last_unstale":"2024-04-26T20:02:44.238882+0000","last_undegraded":"2024-04-26T20:02:44.238882+0000","last_fullsized":"2024-04-26T20:02:44.238882+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:13.957460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.203176+0000","last_change":"2024-04-26T20:02:43.188412+0000","last_active":"2024-04-26T20:02:44.203176+0000","last_peered":"2024-04-26T20:02:44.203176+0000","last_clean":"2024-04-26T20:02:44.203176+0000","last_became_active":"2024-04-26T20:02:43.188160+0000","last_became_peered":"2024-04-26T20:02:43.188160+0000","last_unstale":"2024-04-26T20:02:44.203176+0000","last_undegraded":"2024-04-26T20:02:44.203176+0000","last_fullsized":"2024-04-26T20:02:44.203176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:29.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229456+0000","last_change":"2024-04-26T20:02:44.225429+0000","last_active":"2024-04-26T20:02:44.229456+0000","last_peered":"2024-04-26T20:02:44.229456+0000","last_clean":"2024-04-26T20:02:44.229456+0000","last_became_active":"2024-04-26T20:02:44.225117+0000","last_became_peered":"2024-04-26T20:02:44.225117+0000","last_unstale":"2024-04-26T20:02:44.229456+0000","last_undegraded":"2024-04-26T20:02:44.229456+0000","last_fullsized":"2024-04-26T20:02:44.229456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:23.077372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.200493+0000","last_change":"2024-04-26T20:02:43.202528+0000","last_active":"2024-04-26T20:02:44.200493+0000","last_peered":"2024-04-26T20:02:44.200493+0000","last_clean":"2024-04-26T20:02:44.200493+0000","last_became_active":"2024-04-26T20:02:43.202159+0000","last_became_peered":"2024-04-26T20:02:43.202159+0000","last_unstale":"2024-04-26T20:02:44.200493+0000","last_undegraded":"2024-04-26T20:02:44.200493+0000","last_fullsized":"2024-04-26T20:02:44.200493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:43.562198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226069+0000","last_change":"2024-04-26T20:02:44.218214+0000","last_active":"2024-04-26T20:02:44.226069+0000","last_peered":"2024-04-26T20:02:44.226069+0000","last_clean":"2024-04-26T20:02:44.226069+0000","last_became_active":"2024-04-26T20:02:44.217943+0000","last_became_peered":"2024-04-26T20:02:44.217943+0000","last_unstale":"2024-04-26T20:02:44.226069+0000","last_undegraded":"2024-04-26T20:02:44.226069+0000","last_fullsized":"2024-04-26T20:02:44.226069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:12.333501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.183891+0000","last_change":"2024-04-26T20:02:43.191198+0000","last_active":"2024-04-26T20:02:44.183891+0000","last_peered":"2024-04-26T20:02:44.183891+0000","last_clean":"2024-04-26T20:02:44.183891+0000","last_became_active":"2024-04-26T20:02:43.190842+0000","last_became_peered":"2024-04-26T20:02:43.190842+0000","last_unstale":"2024-04-26T20:02:44.183891+0000","last_undegraded":"2024-04-26T20:02:44.183891+0000","last_fullsized":"2024-04-26T20:02:44.183891+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:21.213851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.210074+0000","last_change":"2024-04-26T20:02:44.206191+0000","last_active":"2024-04-26T20:02:44.210074+0000","last_peered":"2024-04-26T20:02:44.210074+0000","last_clean":"2024-04-26T20:02:44.210074+0000","last_became_active":"2024-04-26T20:02:44.205765+0000","last_became_peered":"2024-04-26T20:02:44.205765+0000","last_unstale":"2024-04-26T20:02:44.210074+0000","last_undegraded":"2024-04-26T20:02:44.210074+0000","last_fullsized":"2024-04-26T20:02:44.210074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:53:34.293453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.211384+0000","last_change":"2024-04-26T20:02:43.196365+0000","last_active":"2024-04-26T20:02:44.211384+0000","last_peered":"2024-04-26T20:02:44.211384+0000","last_clean":"2024-04-26T20:02:44.211384+0000","last_became_active":"2024-04-26T20:02:43.195951+0000","last_became_peered":"2024-04-26T20:02:43.195951+0000","last_unstale":"2024-04-26T20:02:44.211384+0000","last_undegraded":"2024-04-26T20:02:44.211384+0000","last_fullsized":"2024-04-26T20:02:44.211384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:38.257220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.211027+0000","last_change":"2024-04-26T20:02:44.209155+0000","last_active":"2024-04-26T20:02:44.211027+0000","last_peered":"2024-04-26T20:02:44.211027+0000","last_clean":"2024-04-26T20:02:44.211027+0000","last_became_active":"2024-04-26T20:02:44.202194+0000","last_became_peered":"2024-04-26T20:02:44.202194+0000","last_unstale":"2024-04-26T20:02:44.211027+0000","last_undegraded":"2024-04-26T20:02:44.211027+0000","last_fullsized":"2024-04-26T20:02:44.211027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:00.994769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.189160+0000","last_change":"2024-04-26T20:02:43.197770+0000","last_active":"2024-04-26T20:02:44.189160+0000","last_peered":"2024-04-26T20:02:44.189160+0000","last_clean":"2024-04-26T20:02:44.189160+0000","last_became_active":"2024-04-26T20:02:43.197058+0000","last_became_peered":"2024-04-26T20:02:43.197058+0000","last_unstale":"2024-04-26T20:02:44.189160+0000","last_undegraded":"2024-04-26T20:02:44.189160+0000","last_fullsized":"2024-04-26T20:02:44.189160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:41.215693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224982+0000","last_change":"2024-04-26T20:02:44.218725+0000","last_active":"2024-04-26T20:02:44.224982+0000","last_peered":"2024-04-26T20:02:44.224982+0000","last_clean":"2024-04-26T20:02:44.224982+0000","last_became_active":"2024-04-26T20:02:44.218390+0000","last_became_peered":"2024-04-26T20:02:44.218390+0000","last_unstale":"2024-04-26T20:02:44.224982+0000","last_undegraded":"2024-04-26T20:02:44.224982+0000","last_fullsized":"2024-04-26T20:02:44.224982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:16.073241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237390+0000","last_change":"2024-04-26T20:02:43.190342+0000","last_active":"2024-04-26T20:02:44.237390+0000","last_peered":"2024-04-26T20:02:44.237390+0000","last_clean":"2024-04-26T20:02:44.237390+0000","last_became_active":"2024-04-26T20:02:43.190037+0000","last_became_peered":"2024-04-26T20:02:43.190037+0000","last_unstale":"2024-04-26T20:02:44.237390+0000","last_undegraded":"2024-04-26T20:02:44.237390+0000","last_fullsized":"2024-04-26T20:02:44.237390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:57.904098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205280+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:02:44.205280+0000","last_peered":"2024-04-26T20:02:44.205280+0000","last_clean":"2024-04-26T20:02:44.205280+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:02:44.205280+0000","last_undegraded":"2024-04-26T20:02:44.205280+0000","last_fullsized":"2024-04-26T20:02:44.205280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28348,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343492,"statfs":{"total":96636764160,"available":96607735808,"internally_reserved":0,"allocated":1355776,"data_stored":912804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":321912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":757760,"data_stored":321910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":757760,"data_stored":321890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":741376,"data_stored":315045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":323432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":324048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":912188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:48.639 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:02:48.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.872+0000 7fcaf2800640 1 Processor -- start 2024-04-26T20:02:48.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.872+0000 7fcaf2800640 1 -- start start 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0b5660 0x7fcaec06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf2800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcaec0b5d40 0x7fcaec06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf2800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0727f0 con 0x7fcaec0d0e70 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf2800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0b4a10 con 0x7fcaec0b5d40 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0b4cf0 con 0x7fcaec0b5660 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf1e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcaec0b5d40 0x7fcaec06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf1e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fcaec0b5d40 0x7fcaec06e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55898/0 (socket says 172.21.15.88:55898) 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf1e00640 1 -- 172.21.15.88:0/2143973136 learned_addr learned my addr 172.21.15.88:0/2143973136 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:48.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.874+0000 7fcaf0a00640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0b5660 0x7fcaec06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.875+0000 7fcaf1e00640 1 -- 172.21.15.88:0/2143973136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0b5660 msgr2=0x7fcaec06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:48.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.875+0000 7fcaf1e00640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0b5660 0x7fcaec06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.875+0000 7fcaf1e00640 1 -- 172.21.15.88:0/2143973136 >> v1:172.21.15.132:6789/0 conn(0x7fcaec0d0e70 legacy=0x7fcaec0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:48.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.875+0000 7fcaf1e00640 1 -- 172.21.15.88:0/2143973136 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0b4fd0 con 0x7fcaec0b5d40 2024-04-26T20:02:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.876+0000 7fcaf1e00640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5d40 0x7fcaec06e930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009e50 tx=0x7fcae402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ba45b3af855413 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.876+0000 7fcaebe00640 1 -- 172.21.15.88:0/2143973136 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4036070 con 0x7fcaec0b5d40 2024-04-26T20:02:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.876+0000 7fcaebe00640 1 -- 172.21.15.88:0/2143973136 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4031600 con 0x7fcaec0b5d40 2024-04-26T20:02:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.876+0000 7fcaebe00640 1 -- 172.21.15.88:0/2143973136 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40319d0 con 0x7fcaec0b5d40 2024-04-26T20:02:48.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 -- 172.21.15.88:0/2143973136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5d40 msgr2=0x7fcaec06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5d40 0x7fcaec06e930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009e50 tx=0x7fcae402a630 comp rx=0 tx=0).stop 2024-04-26T20:02:48.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 -- 172.21.15.88:0/2143973136 shutdown_connections 2024-04-26T20:02:48.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5d40 0x7fcaec06e930 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 --2- 172.21.15.88:0/2143973136 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0b5660 0x7fcaec06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 -- 172.21.15.88:0/2143973136 >> 172.21.15.88:0/2143973136 conn(0x7fcaec06c740 msgr2=0x7fcaec06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:48.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 -- 172.21.15.88:0/2143973136 shutdown_connections 2024-04-26T20:02:48.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.877+0000 7fcaf2800640 1 -- 172.21.15.88:0/2143973136 wait complete. 2024-04-26T20:02:48.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.879+0000 7fcaf2800640 1 Processor -- start 2024-04-26T20:02:48.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.880+0000 7fcaf2800640 1 -- start start 2024-04-26T20:02:48.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 0x7fcaec0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0d0e70 0x7fcaec14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec15f320 con 0x7fcaec0b5660 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec15f1a0 con 0x7fcaec0b5d40 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf2800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec06ee70 con 0x7fcaec0d0e70 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55914/0 (socket says 172.21.15.88:55914) 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.881+0000 7fcaf1400640 1 -- 172.21.15.88:0/912578748 learned_addr learned my addr 172.21.15.88:0/912578748 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf0a00640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 0x7fcaec0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1e00640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0d0e70 0x7fcaec14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 -- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0d0e70 msgr2=0x7fcaec14e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0d0e70 0x7fcaec14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 -- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 msgr2=0x7fcaec0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 0x7fcaec0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 -- 172.21.15.88:0/912578748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec160fc0 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf0a00640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 0x7fcaec0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf1400640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b420 tx=0x7fcae000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0011830 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec1612a0 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae0016070 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.882+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0011c10 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.883+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec161860 con 0x7fcaec0b5660 2024-04-26T20:02:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.884+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec161d50 con 0x7fcaec0b5660 2024-04-26T20:02:48.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.885+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fcae001b460 con 0x7fcaec0b5660 2024-04-26T20:02:48.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.887+0000 7fcae9600640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:48.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.887+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fcae008b7e0 con 0x7fcaec0b5660 2024-04-26T20:02:48.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.887+0000 7fcaf0a00640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:48.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.887+0000 7fcaf0a00640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 0x7fcacc06ee40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcaec157680 tx=0x7fcad8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:48.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:48.892+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcae0020020 con 0x7fcaec0b5660 2024-04-26T20:02:49.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.090+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcaec0b3830 con 0x7fcacc06c980 2024-04-26T20:02:49.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.109+0000 7fcae9600640 1 -- 172.21.15.88:0/912578748 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251999 (secure 0 0 0) 0x7fcaec0b3830 con 0x7fcacc06c980 2024-04-26T20:02:49.113 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:49.114 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:02:49.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.114+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 msgr2=0x7fcacc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.114+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 0x7fcacc06ee40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcaec157680 tx=0x7fcad8008040 comp rx=0 tx=0).stop 2024-04-26T20:02:49.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.114+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 msgr2=0x7fcaec160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.114+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcae000b420 tx=0x7fcae000b8e0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 shutdown_connections 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fcacc06c980 0x7fcacc06ee40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fcaec0d0e70 0x7fcaec14e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcaec0b5d40 0x7fcaec0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 --2- 172.21.15.88:0/912578748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaec0b5660 0x7fcaec160850 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 >> 172.21.15.88:0/912578748 conn(0x7fcaec06c740 msgr2=0x7fcaec153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 shutdown_connections 2024-04-26T20:02:49.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.115+0000 7fcaf2800640 1 -- 172.21.15.88:0/912578748 wait complete. 2024-04-26T20:02:49.135 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-26T20:02:49.007325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235768,"kb_used_data":7092,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738952,"statfs":{"total":773094113280,"available":772852686848,"internally_reserved":0,"allocated":7262208,"data_stored":3753229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001660"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242179+0000","last_change":"2024-04-26T20:02:44.231595+0000","last_active":"2024-04-26T20:02:44.242179+0000","last_peered":"2024-04-26T20:02:44.242179+0000","last_clean":"2024-04-26T20:02:44.242179+0000","last_became_active":"2024-04-26T20:02:44.227391+0000","last_became_peered":"2024-04-26T20:02:44.227391+0000","last_unstale":"2024-04-26T20:02:44.242179+0000","last_undegraded":"2024-04-26T20:02:44.242179+0000","last_fullsized":"2024-04-26T20:02:44.242179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:43:38.763953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224039+0000","last_change":"2024-04-26T20:02:44.216625+0000","last_active":"2024-04-26T20:02:44.224039+0000","last_peered":"2024-04-26T20:02:44.224039+0000","last_clean":"2024-04-26T20:02:44.224039+0000","last_became_active":"2024-04-26T20:02:44.215725+0000","last_became_peered":"2024-04-26T20:02:44.215725+0000","last_unstale":"2024-04-26T20:02:44.224039+0000","last_undegraded":"2024-04-26T20:02:44.224039+0000","last_fullsized":"2024-04-26T20:02:44.224039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:36.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224457+0000","last_change":"2024-04-26T20:02:44.213989+0000","last_active":"2024-04-26T20:02:44.224457+0000","last_peered":"2024-04-26T20:02:44.224457+0000","last_clean":"2024-04-26T20:02:44.224457+0000","last_became_active":"2024-04-26T20:02:44.213685+0000","last_became_peered":"2024-04-26T20:02:44.213685+0000","last_unstale":"2024-04-26T20:02:44.224457+0000","last_undegraded":"2024-04-26T20:02:44.224457+0000","last_fullsized":"2024-04-26T20:02:44.224457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:13:03.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205699+0000","last_change":"2024-04-26T20:02:44.202223+0000","last_active":"2024-04-26T20:02:44.205699+0000","last_peered":"2024-04-26T20:02:44.205699+0000","last_clean":"2024-04-26T20:02:44.205699+0000","last_became_active":"2024-04-26T20:02:44.201948+0000","last_became_peered":"2024-04-26T20:02:44.201948+0000","last_unstale":"2024-04-26T20:02:44.205699+0000","last_undegraded":"2024-04-26T20:02:44.205699+0000","last_fullsized":"2024-04-26T20:02:44.205699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.031320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.218705+0000","last_change":"2024-04-26T20:02:44.215048+0000","last_active":"2024-04-26T20:02:44.218705+0000","last_peered":"2024-04-26T20:02:44.218705+0000","last_clean":"2024-04-26T20:02:44.218705+0000","last_became_active":"2024-04-26T20:02:44.214743+0000","last_became_peered":"2024-04-26T20:02:44.214743+0000","last_unstale":"2024-04-26T20:02:44.218705+0000","last_undegraded":"2024-04-26T20:02:44.218705+0000","last_fullsized":"2024-04-26T20:02:44.218705+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:20:26.109862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225672+0000","last_change":"2024-04-26T20:02:44.213192+0000","last_active":"2024-04-26T20:02:44.225672+0000","last_peered":"2024-04-26T20:02:44.225672+0000","last_clean":"2024-04-26T20:02:44.225672+0000","last_became_active":"2024-04-26T20:02:44.212831+0000","last_became_peered":"2024-04-26T20:02:44.212831+0000","last_unstale":"2024-04-26T20:02:44.225672+0000","last_undegraded":"2024-04-26T20:02:44.225672+0000","last_fullsized":"2024-04-26T20:02:44.225672+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:12.438185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.210232+0000","last_change":"2024-04-26T20:02:44.207071+0000","last_active":"2024-04-26T20:02:44.210232+0000","last_peered":"2024-04-26T20:02:44.210232+0000","last_clean":"2024-04-26T20:02:44.210232+0000","last_became_active":"2024-04-26T20:02:44.206006+0000","last_became_peered":"2024-04-26T20:02:44.206006+0000","last_unstale":"2024-04-26T20:02:44.210232+0000","last_undegraded":"2024-04-26T20:02:44.210232+0000","last_fullsized":"2024-04-26T20:02:44.210232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:19.045556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223985+0000","last_change":"2024-04-26T20:02:44.211050+0000","last_active":"2024-04-26T20:02:44.223985+0000","last_peered":"2024-04-26T20:02:44.223985+0000","last_clean":"2024-04-26T20:02:44.223985+0000","last_became_active":"2024-04-26T20:02:44.210140+0000","last_became_peered":"2024-04-26T20:02:44.210140+0000","last_unstale":"2024-04-26T20:02:44.223985+0000","last_undegraded":"2024-04-26T20:02:44.223985+0000","last_fullsized":"2024-04-26T20:02:44.223985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:11.659616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235046+0000","last_change":"2024-04-26T20:02:44.232299+0000","last_active":"2024-04-26T20:02:44.235046+0000","last_peered":"2024-04-26T20:02:44.235046+0000","last_clean":"2024-04-26T20:02:44.235046+0000","last_became_active":"2024-04-26T20:02:44.230283+0000","last_became_peered":"2024-04-26T20:02:44.230283+0000","last_unstale":"2024-04-26T20:02:44.235046+0000","last_undegraded":"2024-04-26T20:02:44.235046+0000","last_fullsized":"2024-04-26T20:02:44.235046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:35.436478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235642+0000","last_change":"2024-04-26T20:02:44.231146+0000","last_active":"2024-04-26T20:02:44.235642+0000","last_peered":"2024-04-26T20:02:44.235642+0000","last_clean":"2024-04-26T20:02:44.235642+0000","last_became_active":"2024-04-26T20:02:44.230806+0000","last_became_peered":"2024-04-26T20:02:44.230806+0000","last_unstale":"2024-04-26T20:02:44.235642+0000","last_undegraded":"2024-04-26T20:02:44.235642+0000","last_fullsized":"2024-04-26T20:02:44.235642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:45.730488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237109+0000","last_change":"2024-04-26T20:02:44.233249+0000","last_active":"2024-04-26T20:02:44.237109+0000","last_peered":"2024-04-26T20:02:44.237109+0000","last_clean":"2024-04-26T20:02:44.237109+0000","last_became_active":"2024-04-26T20:02:44.229290+0000","last_became_peered":"2024-04-26T20:02:44.229290+0000","last_unstale":"2024-04-26T20:02:44.237109+0000","last_undegraded":"2024-04-26T20:02:44.237109+0000","last_fullsized":"2024-04-26T20:02:44.237109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:51.840715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.232877+0000","last_change":"2024-04-26T20:02:44.228379+0000","last_active":"2024-04-26T20:02:44.232877+0000","last_peered":"2024-04-26T20:02:44.232877+0000","last_clean":"2024-04-26T20:02:44.232877+0000","last_became_active":"2024-04-26T20:02:44.228061+0000","last_became_peered":"2024-04-26T20:02:44.228061+0000","last_unstale":"2024-04-26T20:02:44.232877+0000","last_undegraded":"2024-04-26T20:02:44.232877+0000","last_fullsized":"2024-04-26T20:02:44.232877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:18:45.730488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.217938+0000","last_change":"2024-04-26T20:02:44.211159+0000","last_active":"2024-04-26T20:02:44.217938+0000","last_peered":"2024-04-26T20:02:44.217938+0000","last_clean":"2024-04-26T20:02:44.217938+0000","last_became_active":"2024-04-26T20:02:44.210822+0000","last_became_peered":"2024-04-26T20:02:44.210822+0000","last_unstale":"2024-04-26T20:02:44.217938+0000","last_undegraded":"2024-04-26T20:02:44.217938+0000","last_fullsized":"2024-04-26T20:02:44.217938+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:58.616520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234623+0000","last_change":"2024-04-26T20:02:44.228548+0000","last_active":"2024-04-26T20:02:44.234623+0000","last_peered":"2024-04-26T20:02:44.234623+0000","last_clean":"2024-04-26T20:02:44.234623+0000","last_became_active":"2024-04-26T20:02:44.228270+0000","last_became_peered":"2024-04-26T20:02:44.228270+0000","last_unstale":"2024-04-26T20:02:44.234623+0000","last_undegraded":"2024-04-26T20:02:44.234623+0000","last_fullsized":"2024-04-26T20:02:44.234623+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:09.664805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.219713+0000","last_change":"2024-04-26T20:02:44.214877+0000","last_active":"2024-04-26T20:02:44.219713+0000","last_peered":"2024-04-26T20:02:44.219713+0000","last_clean":"2024-04-26T20:02:44.219713+0000","last_became_active":"2024-04-26T20:02:44.214509+0000","last_became_peered":"2024-04-26T20:02:44.214509+0000","last_unstale":"2024-04-26T20:02:44.219713+0000","last_undegraded":"2024-04-26T20:02:44.219713+0000","last_fullsized":"2024-04-26T20:02:44.219713+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:42.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228847+0000","last_change":"2024-04-26T20:02:44.220061+0000","last_active":"2024-04-26T20:02:44.228847+0000","last_peered":"2024-04-26T20:02:44.228847+0000","last_clean":"2024-04-26T20:02:44.228847+0000","last_became_active":"2024-04-26T20:02:44.219714+0000","last_became_peered":"2024-04-26T20:02:44.219714+0000","last_unstale":"2024-04-26T20:02:44.228847+0000","last_undegraded":"2024-04-26T20:02:44.228847+0000","last_fullsized":"2024-04-26T20:02:44.228847+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:54.369312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235022+0000","last_change":"2024-04-26T20:02:44.232167+0000","last_active":"2024-04-26T20:02:44.235022+0000","last_peered":"2024-04-26T20:02:44.235022+0000","last_clean":"2024-04-26T20:02:44.235022+0000","last_became_active":"2024-04-26T20:02:44.230634+0000","last_became_peered":"2024-04-26T20:02:44.230634+0000","last_unstale":"2024-04-26T20:02:44.235022+0000","last_undegraded":"2024-04-26T20:02:44.235022+0000","last_fullsized":"2024-04-26T20:02:44.235022+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:35.852379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205363+0000","last_change":"2024-04-26T20:02:44.199914+0000","last_active":"2024-04-26T20:02:44.205363+0000","last_peered":"2024-04-26T20:02:44.205363+0000","last_clean":"2024-04-26T20:02:44.205363+0000","last_became_active":"2024-04-26T20:02:44.199049+0000","last_became_peered":"2024-04-26T20:02:44.199049+0000","last_unstale":"2024-04-26T20:02:44.205363+0000","last_undegraded":"2024-04-26T20:02:44.205363+0000","last_fullsized":"2024-04-26T20:02:44.205363+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:39.710138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226700+0000","last_change":"2024-04-26T20:02:44.218491+0000","last_active":"2024-04-26T20:02:44.226700+0000","last_peered":"2024-04-26T20:02:44.226700+0000","last_clean":"2024-04-26T20:02:44.226700+0000","last_became_active":"2024-04-26T20:02:44.213787+0000","last_became_peered":"2024-04-26T20:02:44.213787+0000","last_unstale":"2024-04-26T20:02:44.226700+0000","last_undegraded":"2024-04-26T20:02:44.226700+0000","last_fullsized":"2024-04-26T20:02:44.226700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:39.271238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.218935+0000","last_change":"2024-04-26T20:02:44.210984+0000","last_active":"2024-04-26T20:02:44.218935+0000","last_peered":"2024-04-26T20:02:44.218935+0000","last_clean":"2024-04-26T20:02:44.218935+0000","last_became_active":"2024-04-26T20:02:44.210665+0000","last_became_peered":"2024-04-26T20:02:44.210665+0000","last_unstale":"2024-04-26T20:02:44.218935+0000","last_undegraded":"2024-04-26T20:02:44.218935+0000","last_fullsized":"2024-04-26T20:02:44.218935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:47.950924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.221886+0000","last_change":"2024-04-26T20:02:44.218084+0000","last_active":"2024-04-26T20:02:44.221886+0000","last_peered":"2024-04-26T20:02:44.221886+0000","last_clean":"2024-04-26T20:02:44.221886+0000","last_became_active":"2024-04-26T20:02:44.217714+0000","last_became_peered":"2024-04-26T20:02:44.217714+0000","last_unstale":"2024-04-26T20:02:44.221886+0000","last_undegraded":"2024-04-26T20:02:44.221886+0000","last_fullsized":"2024-04-26T20:02:44.221886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:02.842485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235632+0000","last_change":"2024-04-26T20:02:44.228336+0000","last_active":"2024-04-26T20:02:44.235632+0000","last_peered":"2024-04-26T20:02:44.235632+0000","last_clean":"2024-04-26T20:02:44.235632+0000","last_became_active":"2024-04-26T20:02:44.228031+0000","last_became_peered":"2024-04-26T20:02:44.228031+0000","last_unstale":"2024-04-26T20:02:44.235632+0000","last_undegraded":"2024-04-26T20:02:44.235632+0000","last_fullsized":"2024-04-26T20:02:44.235632+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:52:06.221711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228863+0000","last_change":"2024-04-26T20:02:44.218940+0000","last_active":"2024-04-26T20:02:44.228863+0000","last_peered":"2024-04-26T20:02:44.228863+0000","last_clean":"2024-04-26T20:02:44.228863+0000","last_became_active":"2024-04-26T20:02:44.218673+0000","last_became_peered":"2024-04-26T20:02:44.218673+0000","last_unstale":"2024-04-26T20:02:44.228863+0000","last_undegraded":"2024-04-26T20:02:44.228863+0000","last_fullsized":"2024-04-26T20:02:44.228863+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:15.492197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.239290+0000","last_change":"2024-04-26T20:02:44.236625+0000","last_active":"2024-04-26T20:02:44.239290+0000","last_peered":"2024-04-26T20:02:44.239290+0000","last_clean":"2024-04-26T20:02:44.239290+0000","last_became_active":"2024-04-26T20:02:44.233643+0000","last_became_peered":"2024-04-26T20:02:44.233643+0000","last_unstale":"2024-04-26T20:02:44.239290+0000","last_undegraded":"2024-04-26T20:02:44.239290+0000","last_fullsized":"2024-04-26T20:02:44.239290+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:17.838609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234915+0000","last_change":"2024-04-26T20:02:44.227643+0000","last_active":"2024-04-26T20:02:44.234915+0000","last_peered":"2024-04-26T20:02:44.234915+0000","last_clean":"2024-04-26T20:02:44.234915+0000","last_became_active":"2024-04-26T20:02:44.227376+0000","last_became_peered":"2024-04-26T20:02:44.227376+0000","last_unstale":"2024-04-26T20:02:44.234915+0000","last_undegraded":"2024-04-26T20:02:44.234915+0000","last_fullsized":"2024-04-26T20:02:44.234915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:39.710138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228451+0000","last_change":"2024-04-26T20:02:44.219382+0000","last_active":"2024-04-26T20:02:44.228451+0000","last_peered":"2024-04-26T20:02:44.228451+0000","last_clean":"2024-04-26T20:02:44.228451+0000","last_became_active":"2024-04-26T20:02:44.219064+0000","last_became_peered":"2024-04-26T20:02:44.219064+0000","last_unstale":"2024-04-26T20:02:44.228451+0000","last_undegraded":"2024-04-26T20:02:44.228451+0000","last_fullsized":"2024-04-26T20:02:44.228451+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:07:50.846924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.231406+0000","last_change":"2024-04-26T20:02:44.227502+0000","last_active":"2024-04-26T20:02:44.231406+0000","last_peered":"2024-04-26T20:02:44.231406+0000","last_clean":"2024-04-26T20:02:44.231406+0000","last_became_active":"2024-04-26T20:02:44.227208+0000","last_became_peered":"2024-04-26T20:02:44.227208+0000","last_unstale":"2024-04-26T20:02:44.231406+0000","last_undegraded":"2024-04-26T20:02:44.231406+0000","last_fullsized":"2024-04-26T20:02:44.231406+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:07:39.824363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235124+0000","last_change":"2024-04-26T20:02:44.228350+0000","last_active":"2024-04-26T20:02:44.235124+0000","last_peered":"2024-04-26T20:02:44.235124+0000","last_clean":"2024-04-26T20:02:44.235124+0000","last_became_active":"2024-04-26T20:02:44.228018+0000","last_became_peered":"2024-04-26T20:02:44.228018+0000","last_unstale":"2024-04-26T20:02:44.235124+0000","last_undegraded":"2024-04-26T20:02:44.235124+0000","last_fullsized":"2024-04-26T20:02:44.235124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:53:34.293453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223057+0000","last_change":"2024-04-26T20:02:44.210322+0000","last_active":"2024-04-26T20:02:44.223057+0000","last_peered":"2024-04-26T20:02:44.223057+0000","last_clean":"2024-04-26T20:02:44.223057+0000","last_became_active":"2024-04-26T20:02:44.207157+0000","last_became_peered":"2024-04-26T20:02:44.207157+0000","last_unstale":"2024-04-26T20:02:44.223057+0000","last_undegraded":"2024-04-26T20:02:44.223057+0000","last_fullsized":"2024-04-26T20:02:44.223057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:28.540865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228886+0000","last_change":"2024-04-26T20:02:44.219921+0000","last_active":"2024-04-26T20:02:44.228886+0000","last_peered":"2024-04-26T20:02:44.228886+0000","last_clean":"2024-04-26T20:02:44.228886+0000","last_became_active":"2024-04-26T20:02:44.219646+0000","last_became_peered":"2024-04-26T20:02:44.219646+0000","last_unstale":"2024-04-26T20:02:44.228886+0000","last_undegraded":"2024-04-26T20:02:44.228886+0000","last_fullsized":"2024-04-26T20:02:44.228886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:52.177288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229006+0000","last_change":"2024-04-26T20:02:44.220753+0000","last_active":"2024-04-26T20:02:44.229006+0000","last_peered":"2024-04-26T20:02:44.229006+0000","last_clean":"2024-04-26T20:02:44.229006+0000","last_became_active":"2024-04-26T20:02:44.220101+0000","last_became_peered":"2024-04-26T20:02:44.220101+0000","last_unstale":"2024-04-26T20:02:44.229006+0000","last_undegraded":"2024-04-26T20:02:44.229006+0000","last_fullsized":"2024-04-26T20:02:44.229006+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:57.650660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223943+0000","last_change":"2024-04-26T20:02:44.209633+0000","last_active":"2024-04-26T20:02:44.223943+0000","last_peered":"2024-04-26T20:02:44.223943+0000","last_clean":"2024-04-26T20:02:44.223943+0000","last_became_active":"2024-04-26T20:02:44.202200+0000","last_became_peered":"2024-04-26T20:02:44.202200+0000","last_unstale":"2024-04-26T20:02:44.223943+0000","last_undegraded":"2024-04-26T20:02:44.223943+0000","last_fullsized":"2024-04-26T20:02:44.223943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:14.948053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237073+0000","last_change":"2024-04-26T20:02:44.231591+0000","last_active":"2024-04-26T20:02:44.237073+0000","last_peered":"2024-04-26T20:02:44.237073+0000","last_clean":"2024-04-26T20:02:44.237073+0000","last_became_active":"2024-04-26T20:02:44.229473+0000","last_became_peered":"2024-04-26T20:02:44.229473+0000","last_unstale":"2024-04-26T20:02:44.237073+0000","last_undegraded":"2024-04-26T20:02:44.237073+0000","last_fullsized":"2024-04-26T20:02:44.237073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:42.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.247242+0000","last_change":"2024-04-26T20:02:43.202527+0000","last_active":"2024-04-26T20:02:44.247242+0000","last_peered":"2024-04-26T20:02:44.247242+0000","last_clean":"2024-04-26T20:02:44.247242+0000","last_became_active":"2024-04-26T20:02:43.202216+0000","last_became_peered":"2024-04-26T20:02:43.202216+0000","last_unstale":"2024-04-26T20:02:44.247242+0000","last_undegraded":"2024-04-26T20:02:44.247242+0000","last_fullsized":"2024-04-26T20:02:44.247242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:29.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237061+0000","last_change":"2024-04-26T20:02:44.232411+0000","last_active":"2024-04-26T20:02:44.237061+0000","last_peered":"2024-04-26T20:02:44.237061+0000","last_clean":"2024-04-26T20:02:44.237061+0000","last_became_active":"2024-04-26T20:02:44.229769+0000","last_became_peered":"2024-04-26T20:02:44.229769+0000","last_unstale":"2024-04-26T20:02:44.237061+0000","last_undegraded":"2024-04-26T20:02:44.237061+0000","last_fullsized":"2024-04-26T20:02:44.237061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:58.358025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234723+0000","last_change":"2024-04-26T20:02:43.179684+0000","last_active":"2024-04-26T20:02:44.234723+0000","last_peered":"2024-04-26T20:02:44.234723+0000","last_clean":"2024-04-26T20:02:44.234723+0000","last_became_active":"2024-04-26T20:02:43.179299+0000","last_became_peered":"2024-04-26T20:02:43.179299+0000","last_unstale":"2024-04-26T20:02:44.234723+0000","last_undegraded":"2024-04-26T20:02:44.234723+0000","last_fullsized":"2024-04-26T20:02:44.234723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:26:14.478180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235760+0000","last_change":"2024-04-26T20:02:44.221688+0000","last_active":"2024-04-26T20:02:44.235760+0000","last_peered":"2024-04-26T20:02:44.235760+0000","last_clean":"2024-04-26T20:02:44.235760+0000","last_became_active":"2024-04-26T20:02:44.221367+0000","last_became_peered":"2024-04-26T20:02:44.221367+0000","last_unstale":"2024-04-26T20:02:44.235760+0000","last_undegraded":"2024-04-26T20:02:44.235760+0000","last_fullsized":"2024-04-26T20:02:44.235760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:47.708744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236502+0000","last_change":"2024-04-26T20:02:43.202054+0000","last_active":"2024-04-26T20:02:44.236502+0000","last_peered":"2024-04-26T20:02:44.236502+0000","last_clean":"2024-04-26T20:02:44.236502+0000","last_became_active":"2024-04-26T20:02:43.201768+0000","last_became_peered":"2024-04-26T20:02:43.201768+0000","last_unstale":"2024-04-26T20:02:44.236502+0000","last_undegraded":"2024-04-26T20:02:44.236502+0000","last_fullsized":"2024-04-26T20:02:44.236502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:46.694726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235511+0000","last_change":"2024-04-26T20:02:44.225032+0000","last_active":"2024-04-26T20:02:44.235511+0000","last_peered":"2024-04-26T20:02:44.235511+0000","last_clean":"2024-04-26T20:02:44.235511+0000","last_became_active":"2024-04-26T20:02:44.224669+0000","last_became_peered":"2024-04-26T20:02:44.224669+0000","last_unstale":"2024-04-26T20:02:44.235511+0000","last_undegraded":"2024-04-26T20:02:44.235511+0000","last_fullsized":"2024-04-26T20:02:44.235511+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:47.950924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237580+0000","last_change":"2024-04-26T20:02:43.190355+0000","last_active":"2024-04-26T20:02:44.237580+0000","last_peered":"2024-04-26T20:02:44.237580+0000","last_clean":"2024-04-26T20:02:44.237580+0000","last_became_active":"2024-04-26T20:02:43.190059+0000","last_became_peered":"2024-04-26T20:02:43.190059+0000","last_unstale":"2024-04-26T20:02:44.237580+0000","last_undegraded":"2024-04-26T20:02:44.237580+0000","last_fullsized":"2024-04-26T20:02:44.237580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:38.176561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.207961+0000","last_change":"2024-04-26T20:02:44.204413+0000","last_active":"2024-04-26T20:02:44.207961+0000","last_peered":"2024-04-26T20:02:44.207961+0000","last_clean":"2024-04-26T20:02:44.207961+0000","last_became_active":"2024-04-26T20:02:44.203972+0000","last_became_peered":"2024-04-26T20:02:44.203972+0000","last_unstale":"2024-04-26T20:02:44.207961+0000","last_undegraded":"2024-04-26T20:02:44.207961+0000","last_fullsized":"2024-04-26T20:02:44.207961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:25.653238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226896+0000","last_change":"2024-04-26T20:02:43.192634+0000","last_active":"2024-04-26T20:02:44.226896+0000","last_peered":"2024-04-26T20:02:44.226896+0000","last_clean":"2024-04-26T20:02:44.226896+0000","last_became_active":"2024-04-26T20:02:43.192306+0000","last_became_peered":"2024-04-26T20:02:43.192306+0000","last_unstale":"2024-04-26T20:02:44.226896+0000","last_undegraded":"2024-04-26T20:02:44.226896+0000","last_fullsized":"2024-04-26T20:02:44.226896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:36.450019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235161+0000","last_change":"2024-04-26T20:02:44.228277+0000","last_active":"2024-04-26T20:02:44.235161+0000","last_peered":"2024-04-26T20:02:44.235161+0000","last_clean":"2024-04-26T20:02:44.235161+0000","last_became_active":"2024-04-26T20:02:44.227905+0000","last_became_peered":"2024-04-26T20:02:44.227905+0000","last_unstale":"2024-04-26T20:02:44.235161+0000","last_undegraded":"2024-04-26T20:02:44.235161+0000","last_fullsized":"2024-04-26T20:02:44.235161+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:53.031320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.235939+0000","last_change":"2024-04-26T20:02:43.192717+0000","last_active":"2024-04-26T20:02:44.235939+0000","last_peered":"2024-04-26T20:02:44.235939+0000","last_clean":"2024-04-26T20:02:44.235939+0000","last_became_active":"2024-04-26T20:02:43.192413+0000","last_became_peered":"2024-04-26T20:02:43.192413+0000","last_unstale":"2024-04-26T20:02:44.235939+0000","last_undegraded":"2024-04-26T20:02:44.235939+0000","last_fullsized":"2024-04-26T20:02:44.235939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:04.201771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.220445+0000","last_change":"2024-04-26T20:02:44.217953+0000","last_active":"2024-04-26T20:02:44.220445+0000","last_peered":"2024-04-26T20:02:44.220445+0000","last_clean":"2024-04-26T20:02:44.220445+0000","last_became_active":"2024-04-26T20:02:44.217548+0000","last_became_peered":"2024-04-26T20:02:44.217548+0000","last_unstale":"2024-04-26T20:02:44.220445+0000","last_undegraded":"2024-04-26T20:02:44.220445+0000","last_fullsized":"2024-04-26T20:02:44.220445+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:58.286972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.183862+0000","last_change":"2024-04-26T20:02:43.194019+0000","last_active":"2024-04-26T20:02:44.183862+0000","last_peered":"2024-04-26T20:02:44.183862+0000","last_clean":"2024-04-26T20:02:44.183862+0000","last_became_active":"2024-04-26T20:02:43.193689+0000","last_became_peered":"2024-04-26T20:02:43.193689+0000","last_unstale":"2024-04-26T20:02:44.183862+0000","last_undegraded":"2024-04-26T20:02:44.183862+0000","last_fullsized":"2024-04-26T20:02:44.183862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:05.375211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.219691+0000","last_change":"2024-04-26T20:02:44.213286+0000","last_active":"2024-04-26T20:02:44.219691+0000","last_peered":"2024-04-26T20:02:44.219691+0000","last_clean":"2024-04-26T20:02:44.219691+0000","last_became_active":"2024-04-26T20:02:44.212956+0000","last_became_peered":"2024-04-26T20:02:44.212956+0000","last_unstale":"2024-04-26T20:02:44.219691+0000","last_undegraded":"2024-04-26T20:02:44.219691+0000","last_fullsized":"2024-04-26T20:02:44.219691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:44.777171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.204156+0000","last_change":"2024-04-26T20:02:43.179363+0000","last_active":"2024-04-26T20:02:44.204156+0000","last_peered":"2024-04-26T20:02:44.204156+0000","last_clean":"2024-04-26T20:02:44.204156+0000","last_became_active":"2024-04-26T20:02:43.179053+0000","last_became_peered":"2024-04-26T20:02:43.179053+0000","last_unstale":"2024-04-26T20:02:44.204156+0000","last_undegraded":"2024-04-26T20:02:44.204156+0000","last_fullsized":"2024-04-26T20:02:44.204156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:53.470531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.228424+0000","last_change":"2024-04-26T20:02:44.216999+0000","last_active":"2024-04-26T20:02:44.228424+0000","last_peered":"2024-04-26T20:02:44.228424+0000","last_clean":"2024-04-26T20:02:44.228424+0000","last_became_active":"2024-04-26T20:02:44.216694+0000","last_became_peered":"2024-04-26T20:02:44.216694+0000","last_unstale":"2024-04-26T20:02:44.228424+0000","last_undegraded":"2024-04-26T20:02:44.228424+0000","last_fullsized":"2024-04-26T20:02:44.228424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:56.139921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.212251+0000","last_change":"2024-04-26T20:02:43.188207+0000","last_active":"2024-04-26T20:02:44.212251+0000","last_peered":"2024-04-26T20:02:44.212251+0000","last_clean":"2024-04-26T20:02:44.212251+0000","last_became_active":"2024-04-26T20:02:43.187889+0000","last_became_peered":"2024-04-26T20:02:43.187889+0000","last_unstale":"2024-04-26T20:02:44.212251+0000","last_undegraded":"2024-04-26T20:02:44.212251+0000","last_fullsized":"2024-04-26T20:02:44.212251+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:05.375211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242234+0000","last_change":"2024-04-26T20:02:44.236248+0000","last_active":"2024-04-26T20:02:44.242234+0000","last_peered":"2024-04-26T20:02:44.242234+0000","last_clean":"2024-04-26T20:02:44.242234+0000","last_became_active":"2024-04-26T20:02:44.235020+0000","last_became_peered":"2024-04-26T20:02:44.235020+0000","last_unstale":"2024-04-26T20:02:44.242234+0000","last_undegraded":"2024-04-26T20:02:44.242234+0000","last_fullsized":"2024-04-26T20:02:44.242234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:24:37.193478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.196912+0000","last_change":"2024-04-26T20:02:43.198670+0000","last_active":"2024-04-26T20:02:44.196912+0000","last_peered":"2024-04-26T20:02:44.196912+0000","last_clean":"2024-04-26T20:02:44.196912+0000","last_became_active":"2024-04-26T20:02:43.197285+0000","last_became_peered":"2024-04-26T20:02:43.197285+0000","last_unstale":"2024-04-26T20:02:44.196912+0000","last_undegraded":"2024-04-26T20:02:44.196912+0000","last_fullsized":"2024-04-26T20:02:44.196912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:40.535868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224360+0000","last_change":"2024-04-26T20:02:44.214898+0000","last_active":"2024-04-26T20:02:44.224360+0000","last_peered":"2024-04-26T20:02:44.224360+0000","last_clean":"2024-04-26T20:02:44.224360+0000","last_became_active":"2024-04-26T20:02:44.214586+0000","last_became_peered":"2024-04-26T20:02:44.214586+0000","last_unstale":"2024-04-26T20:02:44.224360+0000","last_undegraded":"2024-04-26T20:02:44.224360+0000","last_fullsized":"2024-04-26T20:02:44.224360+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:37.848234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.234386+0000","last_change":"2024-04-26T20:02:43.192534+0000","last_active":"2024-04-26T20:02:44.234386+0000","last_peered":"2024-04-26T20:02:44.234386+0000","last_clean":"2024-04-26T20:02:44.234386+0000","last_became_active":"2024-04-26T20:02:43.191930+0000","last_became_peered":"2024-04-26T20:02:43.191930+0000","last_unstale":"2024-04-26T20:02:44.234386+0000","last_undegraded":"2024-04-26T20:02:44.234386+0000","last_fullsized":"2024-04-26T20:02:44.234386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:18:38.535802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229216+0000","last_change":"2024-04-26T20:02:44.224559+0000","last_active":"2024-04-26T20:02:44.229216+0000","last_peered":"2024-04-26T20:02:44.229216+0000","last_clean":"2024-04-26T20:02:44.229216+0000","last_became_active":"2024-04-26T20:02:44.221905+0000","last_became_peered":"2024-04-26T20:02:44.221905+0000","last_unstale":"2024-04-26T20:02:44.229216+0000","last_undegraded":"2024-04-26T20:02:44.229216+0000","last_fullsized":"2024-04-26T20:02:44.229216+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:33.342047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.239806+0000","last_change":"2024-04-26T20:02:43.179957+0000","last_active":"2024-04-26T20:02:44.239806+0000","last_peered":"2024-04-26T20:02:44.239806+0000","last_clean":"2024-04-26T20:02:44.239806+0000","last_became_active":"2024-04-26T20:02:43.179606+0000","last_became_peered":"2024-04-26T20:02:43.179606+0000","last_unstale":"2024-04-26T20:02:44.239806+0000","last_undegraded":"2024-04-26T20:02:44.239806+0000","last_fullsized":"2024-04-26T20:02:44.239806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:53.355295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236309+0000","last_change":"2024-04-26T20:02:44.225784+0000","last_active":"2024-04-26T20:02:44.236309+0000","last_peered":"2024-04-26T20:02:44.236309+0000","last_clean":"2024-04-26T20:02:44.236309+0000","last_became_active":"2024-04-26T20:02:44.225496+0000","last_became_peered":"2024-04-26T20:02:44.225496+0000","last_unstale":"2024-04-26T20:02:44.236309+0000","last_undegraded":"2024-04-26T20:02:44.236309+0000","last_fullsized":"2024-04-26T20:02:44.236309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:58.616520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237374+0000","last_change":"2024-04-26T20:02:43.192741+0000","last_active":"2024-04-26T20:02:44.237374+0000","last_peered":"2024-04-26T20:02:44.237374+0000","last_clean":"2024-04-26T20:02:44.237374+0000","last_became_active":"2024-04-26T20:02:43.192405+0000","last_became_peered":"2024-04-26T20:02:43.192405+0000","last_unstale":"2024-04-26T20:02:44.237374+0000","last_undegraded":"2024-04-26T20:02:44.237374+0000","last_fullsized":"2024-04-26T20:02:44.237374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:01.828467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229365+0000","last_change":"2024-04-26T20:02:44.217064+0000","last_active":"2024-04-26T20:02:44.229365+0000","last_peered":"2024-04-26T20:02:44.229365+0000","last_clean":"2024-04-26T20:02:44.229365+0000","last_became_active":"2024-04-26T20:02:44.216719+0000","last_became_peered":"2024-04-26T20:02:44.216719+0000","last_unstale":"2024-04-26T20:02:44.229365+0000","last_undegraded":"2024-04-26T20:02:44.229365+0000","last_fullsized":"2024-04-26T20:02:44.229365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:59.254006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191403+0000","last_change":"2024-04-26T20:02:43.198415+0000","last_active":"2024-04-26T20:02:44.191403+0000","last_peered":"2024-04-26T20:02:44.191403+0000","last_clean":"2024-04-26T20:02:44.191403+0000","last_became_active":"2024-04-26T20:02:43.198143+0000","last_became_peered":"2024-04-26T20:02:43.198143+0000","last_unstale":"2024-04-26T20:02:44.191403+0000","last_undegraded":"2024-04-26T20:02:44.191403+0000","last_fullsized":"2024-04-26T20:02:44.191403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:12.625877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224970+0000","last_change":"2024-04-26T20:02:44.222241+0000","last_active":"2024-04-26T20:02:44.224970+0000","last_peered":"2024-04-26T20:02:44.224970+0000","last_clean":"2024-04-26T20:02:44.224970+0000","last_became_active":"2024-04-26T20:02:44.221094+0000","last_became_peered":"2024-04-26T20:02:44.221094+0000","last_unstale":"2024-04-26T20:02:44.224970+0000","last_undegraded":"2024-04-26T20:02:44.224970+0000","last_fullsized":"2024-04-26T20:02:44.224970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:22:51.840715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191896+0000","last_change":"2024-04-26T20:02:43.196404+0000","last_active":"2024-04-26T20:02:44.191896+0000","last_peered":"2024-04-26T20:02:44.191896+0000","last_clean":"2024-04-26T20:02:44.191896+0000","last_became_active":"2024-04-26T20:02:43.196108+0000","last_became_peered":"2024-04-26T20:02:43.196108+0000","last_unstale":"2024-04-26T20:02:44.191896+0000","last_undegraded":"2024-04-26T20:02:44.191896+0000","last_fullsized":"2024-04-26T20:02:44.191896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:20.412508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.231471+0000","last_change":"2024-04-26T20:02:44.224192+0000","last_active":"2024-04-26T20:02:44.231471+0000","last_peered":"2024-04-26T20:02:44.231471+0000","last_clean":"2024-04-26T20:02:44.231471+0000","last_became_active":"2024-04-26T20:02:44.220837+0000","last_became_peered":"2024-04-26T20:02:44.220837+0000","last_unstale":"2024-04-26T20:02:44.231471+0000","last_undegraded":"2024-04-26T20:02:44.231471+0000","last_fullsized":"2024-04-26T20:02:44.231471+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:00.635462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.243044+0000","last_change":"2024-04-26T20:02:43.197576+0000","last_active":"2024-04-26T20:02:44.243044+0000","last_peered":"2024-04-26T20:02:44.243044+0000","last_clean":"2024-04-26T20:02:44.243044+0000","last_became_active":"2024-04-26T20:02:43.197290+0000","last_became_peered":"2024-04-26T20:02:43.197290+0000","last_unstale":"2024-04-26T20:02:44.243044+0000","last_undegraded":"2024-04-26T20:02:44.243044+0000","last_fullsized":"2024-04-26T20:02:44.243044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:35.363409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.217403+0000","last_change":"2024-04-26T20:02:44.215169+0000","last_active":"2024-04-26T20:02:44.217403+0000","last_peered":"2024-04-26T20:02:44.217403+0000","last_clean":"2024-04-26T20:02:44.217403+0000","last_became_active":"2024-04-26T20:02:44.214836+0000","last_became_peered":"2024-04-26T20:02:44.214836+0000","last_unstale":"2024-04-26T20:02:44.217403+0000","last_undegraded":"2024-04-26T20:02:44.217403+0000","last_fullsized":"2024-04-26T20:02:44.217403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:22.227868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238515+0000","last_change":"2024-04-26T20:02:43.199855+0000","last_active":"2024-04-26T20:02:44.238515+0000","last_peered":"2024-04-26T20:02:44.238515+0000","last_clean":"2024-04-26T20:02:44.238515+0000","last_became_active":"2024-04-26T20:02:43.194522+0000","last_became_peered":"2024-04-26T20:02:43.194522+0000","last_unstale":"2024-04-26T20:02:44.238515+0000","last_undegraded":"2024-04-26T20:02:44.238515+0000","last_fullsized":"2024-04-26T20:02:44.238515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:20.072021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.242210+0000","last_change":"2024-04-26T20:02:44.235859+0000","last_active":"2024-04-26T20:02:44.242210+0000","last_peered":"2024-04-26T20:02:44.242210+0000","last_clean":"2024-04-26T20:02:44.242210+0000","last_became_active":"2024-04-26T20:02:44.232674+0000","last_became_peered":"2024-04-26T20:02:44.232674+0000","last_unstale":"2024-04-26T20:02:44.242210+0000","last_undegraded":"2024-04-26T20:02:44.242210+0000","last_fullsized":"2024-04-26T20:02:44.242210+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:56.396247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.236809+0000","last_change":"2024-04-26T20:02:43.201318+0000","last_active":"2024-04-26T20:02:44.236809+0000","last_peered":"2024-04-26T20:02:44.236809+0000","last_clean":"2024-04-26T20:02:44.236809+0000","last_became_active":"2024-04-26T20:02:43.201012+0000","last_became_peered":"2024-04-26T20:02:43.201012+0000","last_unstale":"2024-04-26T20:02:44.236809+0000","last_undegraded":"2024-04-26T20:02:44.236809+0000","last_fullsized":"2024-04-26T20:02:44.236809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:11.319483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225106+0000","last_change":"2024-04-26T20:02:44.218274+0000","last_active":"2024-04-26T20:02:44.225106+0000","last_peered":"2024-04-26T20:02:44.225106+0000","last_clean":"2024-04-26T20:02:44.225106+0000","last_became_active":"2024-04-26T20:02:44.218018+0000","last_became_peered":"2024-04-26T20:02:44.218018+0000","last_unstale":"2024-04-26T20:02:44.225106+0000","last_undegraded":"2024-04-26T20:02:44.225106+0000","last_fullsized":"2024-04-26T20:02:44.225106+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:52:06.221711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237711+0000","last_change":"2024-04-26T20:02:43.196074+0000","last_active":"2024-04-26T20:02:44.237711+0000","last_peered":"2024-04-26T20:02:44.237711+0000","last_clean":"2024-04-26T20:02:44.237711+0000","last_became_active":"2024-04-26T20:02:43.195148+0000","last_became_peered":"2024-04-26T20:02:43.195148+0000","last_unstale":"2024-04-26T20:02:44.237711+0000","last_undegraded":"2024-04-26T20:02:44.237711+0000","last_fullsized":"2024-04-26T20:02:44.237711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:13.940582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.225410+0000","last_change":"2024-04-26T20:02:44.221039+0000","last_active":"2024-04-26T20:02:44.225410+0000","last_peered":"2024-04-26T20:02:44.225410+0000","last_clean":"2024-04-26T20:02:44.225410+0000","last_became_active":"2024-04-26T20:02:44.220749+0000","last_became_peered":"2024-04-26T20:02:44.220749+0000","last_unstale":"2024-04-26T20:02:44.225410+0000","last_undegraded":"2024-04-26T20:02:44.225410+0000","last_fullsized":"2024-04-26T20:02:44.225410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:50:31.133315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.209498+0000","last_change":"2024-04-26T20:02:43.201376+0000","last_active":"2024-04-26T20:02:44.209498+0000","last_peered":"2024-04-26T20:02:44.209498+0000","last_clean":"2024-04-26T20:02:44.209498+0000","last_became_active":"2024-04-26T20:02:43.201092+0000","last_became_peered":"2024-04-26T20:02:43.201092+0000","last_unstale":"2024-04-26T20:02:44.209498+0000","last_undegraded":"2024-04-26T20:02:44.209498+0000","last_fullsized":"2024-04-26T20:02:44.209498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:15.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223084+0000","last_change":"2024-04-26T20:02:44.211768+0000","last_active":"2024-04-26T20:02:44.223084+0000","last_peered":"2024-04-26T20:02:44.223084+0000","last_clean":"2024-04-26T20:02:44.223084+0000","last_became_active":"2024-04-26T20:02:44.211342+0000","last_became_peered":"2024-04-26T20:02:44.211342+0000","last_unstale":"2024-04-26T20:02:44.223084+0000","last_undegraded":"2024-04-26T20:02:44.223084+0000","last_fullsized":"2024-04-26T20:02:44.223084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:22.973327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.190017+0000","last_change":"2024-04-26T20:02:43.191853+0000","last_active":"2024-04-26T20:02:44.190017+0000","last_peered":"2024-04-26T20:02:44.190017+0000","last_clean":"2024-04-26T20:02:44.190017+0000","last_became_active":"2024-04-26T20:02:43.191514+0000","last_became_peered":"2024-04-26T20:02:43.191514+0000","last_unstale":"2024-04-26T20:02:44.190017+0000","last_undegraded":"2024-04-26T20:02:44.190017+0000","last_fullsized":"2024-04-26T20:02:44.190017+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:47:42.640406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224848+0000","last_change":"2024-04-26T20:02:44.216878+0000","last_active":"2024-04-26T20:02:44.224848+0000","last_peered":"2024-04-26T20:02:44.224848+0000","last_clean":"2024-04-26T20:02:44.224848+0000","last_became_active":"2024-04-26T20:02:44.216564+0000","last_became_peered":"2024-04-26T20:02:44.216564+0000","last_unstale":"2024-04-26T20:02:44.224848+0000","last_undegraded":"2024-04-26T20:02:44.224848+0000","last_fullsized":"2024-04-26T20:02:44.224848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:52:05.473333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.192237+0000","last_change":"2024-04-26T20:02:43.201147+0000","last_active":"2024-04-26T20:02:44.192237+0000","last_peered":"2024-04-26T20:02:44.192237+0000","last_clean":"2024-04-26T20:02:44.192237+0000","last_became_active":"2024-04-26T20:02:43.200819+0000","last_became_peered":"2024-04-26T20:02:43.200819+0000","last_unstale":"2024-04-26T20:02:44.192237+0000","last_undegraded":"2024-04-26T20:02:44.192237+0000","last_fullsized":"2024-04-26T20:02:44.192237+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:17.530347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.223118+0000","last_change":"2024-04-26T20:02:44.213364+0000","last_active":"2024-04-26T20:02:44.223118+0000","last_peered":"2024-04-26T20:02:44.223118+0000","last_clean":"2024-04-26T20:02:44.223118+0000","last_became_active":"2024-04-26T20:02:44.212503+0000","last_became_peered":"2024-04-26T20:02:44.212503+0000","last_unstale":"2024-04-26T20:02:44.223118+0000","last_undegraded":"2024-04-26T20:02:44.223118+0000","last_fullsized":"2024-04-26T20:02:44.223118+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:23:15.737350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.201317+0000","last_change":"2024-04-26T20:02:43.179908+0000","last_active":"2024-04-26T20:02:44.201317+0000","last_peered":"2024-04-26T20:02:44.201317+0000","last_clean":"2024-04-26T20:02:44.201317+0000","last_became_active":"2024-04-26T20:02:43.179279+0000","last_became_peered":"2024-04-26T20:02:43.179279+0000","last_unstale":"2024-04-26T20:02:44.201317+0000","last_undegraded":"2024-04-26T20:02:44.201317+0000","last_fullsized":"2024-04-26T20:02:44.201317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:15.142065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.232271+0000","last_change":"2024-04-26T20:02:44.228108+0000","last_active":"2024-04-26T20:02:44.232271+0000","last_peered":"2024-04-26T20:02:44.232271+0000","last_clean":"2024-04-26T20:02:44.232271+0000","last_became_active":"2024-04-26T20:02:44.227819+0000","last_became_peered":"2024-04-26T20:02:44.227819+0000","last_unstale":"2024-04-26T20:02:44.232271+0000","last_undegraded":"2024-04-26T20:02:44.232271+0000","last_fullsized":"2024-04-26T20:02:44.232271+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:13.957460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238317+0000","last_change":"2024-04-26T20:02:43.197861+0000","last_active":"2024-04-26T20:02:44.238317+0000","last_peered":"2024-04-26T20:02:44.238317+0000","last_clean":"2024-04-26T20:02:44.238317+0000","last_became_active":"2024-04-26T20:02:43.197576+0000","last_became_peered":"2024-04-26T20:02:43.197576+0000","last_unstale":"2024-04-26T20:02:44.238317+0000","last_undegraded":"2024-04-26T20:02:44.238317+0000","last_fullsized":"2024-04-26T20:02:44.238317+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:07.448332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.207718+0000","last_change":"2024-04-26T20:02:44.206106+0000","last_active":"2024-04-26T20:02:44.207718+0000","last_peered":"2024-04-26T20:02:44.207718+0000","last_clean":"2024-04-26T20:02:44.207718+0000","last_became_active":"2024-04-26T20:02:44.205794+0000","last_became_peered":"2024-04-26T20:02:44.205794+0000","last_unstale":"2024-04-26T20:02:44.207718+0000","last_undegraded":"2024-04-26T20:02:44.207718+0000","last_fullsized":"2024-04-26T20:02:44.207718+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:43.489102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.191422+0000","last_change":"2024-04-26T20:02:43.193051+0000","last_active":"2024-04-26T20:02:44.191422+0000","last_peered":"2024-04-26T20:02:44.191422+0000","last_clean":"2024-04-26T20:02:44.191422+0000","last_became_active":"2024-04-26T20:02:43.192761+0000","last_became_peered":"2024-04-26T20:02:43.192761+0000","last_unstale":"2024-04-26T20:02:44.191422+0000","last_undegraded":"2024-04-26T20:02:44.191422+0000","last_fullsized":"2024-04-26T20:02:44.191422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:25.562701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237150+0000","last_change":"2024-04-26T20:02:44.231343+0000","last_active":"2024-04-26T20:02:44.237150+0000","last_peered":"2024-04-26T20:02:44.237150+0000","last_clean":"2024-04-26T20:02:44.237150+0000","last_became_active":"2024-04-26T20:02:44.229669+0000","last_became_peered":"2024-04-26T20:02:44.229669+0000","last_unstale":"2024-04-26T20:02:44.237150+0000","last_undegraded":"2024-04-26T20:02:44.237150+0000","last_fullsized":"2024-04-26T20:02:44.237150+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:39:44.777171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.209872+0000","last_change":"2024-04-26T20:02:43.182063+0000","last_active":"2024-04-26T20:02:44.209872+0000","last_peered":"2024-04-26T20:02:44.209872+0000","last_clean":"2024-04-26T20:02:44.209872+0000","last_became_active":"2024-04-26T20:02:43.181661+0000","last_became_peered":"2024-04-26T20:02:43.181661+0000","last_unstale":"2024-04-26T20:02:44.209872+0000","last_undegraded":"2024-04-26T20:02:44.209872+0000","last_fullsized":"2024-04-26T20:02:44.209872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:36.194942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.238882+0000","last_change":"2024-04-26T20:02:44.234787+0000","last_active":"2024-04-26T20:02:44.238882+0000","last_peered":"2024-04-26T20:02:44.238882+0000","last_clean":"2024-04-26T20:02:44.238882+0000","last_became_active":"2024-04-26T20:02:44.233632+0000","last_became_peered":"2024-04-26T20:02:44.233632+0000","last_unstale":"2024-04-26T20:02:44.238882+0000","last_undegraded":"2024-04-26T20:02:44.238882+0000","last_fullsized":"2024-04-26T20:02:44.238882+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:13.957460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.203176+0000","last_change":"2024-04-26T20:02:43.188412+0000","last_active":"2024-04-26T20:02:44.203176+0000","last_peered":"2024-04-26T20:02:44.203176+0000","last_clean":"2024-04-26T20:02:44.203176+0000","last_became_active":"2024-04-26T20:02:43.188160+0000","last_became_peered":"2024-04-26T20:02:43.188160+0000","last_unstale":"2024-04-26T20:02:44.203176+0000","last_undegraded":"2024-04-26T20:02:44.203176+0000","last_fullsized":"2024-04-26T20:02:44.203176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:29.096077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.229456+0000","last_change":"2024-04-26T20:02:44.225429+0000","last_active":"2024-04-26T20:02:44.229456+0000","last_peered":"2024-04-26T20:02:44.229456+0000","last_clean":"2024-04-26T20:02:44.229456+0000","last_became_active":"2024-04-26T20:02:44.225117+0000","last_became_peered":"2024-04-26T20:02:44.225117+0000","last_unstale":"2024-04-26T20:02:44.229456+0000","last_undegraded":"2024-04-26T20:02:44.229456+0000","last_fullsized":"2024-04-26T20:02:44.229456+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:23.077372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.200493+0000","last_change":"2024-04-26T20:02:43.202528+0000","last_active":"2024-04-26T20:02:44.200493+0000","last_peered":"2024-04-26T20:02:44.200493+0000","last_clean":"2024-04-26T20:02:44.200493+0000","last_became_active":"2024-04-26T20:02:43.202159+0000","last_became_peered":"2024-04-26T20:02:43.202159+0000","last_unstale":"2024-04-26T20:02:44.200493+0000","last_undegraded":"2024-04-26T20:02:44.200493+0000","last_fullsized":"2024-04-26T20:02:44.200493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:43.562198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.226069+0000","last_change":"2024-04-26T20:02:44.218214+0000","last_active":"2024-04-26T20:02:44.226069+0000","last_peered":"2024-04-26T20:02:44.226069+0000","last_clean":"2024-04-26T20:02:44.226069+0000","last_became_active":"2024-04-26T20:02:44.217943+0000","last_became_peered":"2024-04-26T20:02:44.217943+0000","last_unstale":"2024-04-26T20:02:44.226069+0000","last_undegraded":"2024-04-26T20:02:44.226069+0000","last_fullsized":"2024-04-26T20:02:44.226069+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:12.333501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.183891+0000","last_change":"2024-04-26T20:02:43.191198+0000","last_active":"2024-04-26T20:02:44.183891+0000","last_peered":"2024-04-26T20:02:44.183891+0000","last_clean":"2024-04-26T20:02:44.183891+0000","last_became_active":"2024-04-26T20:02:43.190842+0000","last_became_peered":"2024-04-26T20:02:43.190842+0000","last_unstale":"2024-04-26T20:02:44.183891+0000","last_undegraded":"2024-04-26T20:02:44.183891+0000","last_fullsized":"2024-04-26T20:02:44.183891+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:21.213851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.210074+0000","last_change":"2024-04-26T20:02:44.206191+0000","last_active":"2024-04-26T20:02:44.210074+0000","last_peered":"2024-04-26T20:02:44.210074+0000","last_clean":"2024-04-26T20:02:44.210074+0000","last_became_active":"2024-04-26T20:02:44.205765+0000","last_became_peered":"2024-04-26T20:02:44.205765+0000","last_unstale":"2024-04-26T20:02:44.210074+0000","last_undegraded":"2024-04-26T20:02:44.210074+0000","last_fullsized":"2024-04-26T20:02:44.210074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:53:34.293453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.211384+0000","last_change":"2024-04-26T20:02:43.196365+0000","last_active":"2024-04-26T20:02:44.211384+0000","last_peered":"2024-04-26T20:02:44.211384+0000","last_clean":"2024-04-26T20:02:44.211384+0000","last_became_active":"2024-04-26T20:02:43.195951+0000","last_became_peered":"2024-04-26T20:02:43.195951+0000","last_unstale":"2024-04-26T20:02:44.211384+0000","last_undegraded":"2024-04-26T20:02:44.211384+0000","last_fullsized":"2024-04-26T20:02:44.211384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:38.257220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.211027+0000","last_change":"2024-04-26T20:02:44.209155+0000","last_active":"2024-04-26T20:02:44.211027+0000","last_peered":"2024-04-26T20:02:44.211027+0000","last_clean":"2024-04-26T20:02:44.211027+0000","last_became_active":"2024-04-26T20:02:44.202194+0000","last_became_peered":"2024-04-26T20:02:44.202194+0000","last_unstale":"2024-04-26T20:02:44.211027+0000","last_undegraded":"2024-04-26T20:02:44.211027+0000","last_fullsized":"2024-04-26T20:02:44.211027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:00.994769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.189160+0000","last_change":"2024-04-26T20:02:43.197770+0000","last_active":"2024-04-26T20:02:44.189160+0000","last_peered":"2024-04-26T20:02:44.189160+0000","last_clean":"2024-04-26T20:02:44.189160+0000","last_became_active":"2024-04-26T20:02:43.197058+0000","last_became_peered":"2024-04-26T20:02:43.197058+0000","last_unstale":"2024-04-26T20:02:44.189160+0000","last_undegraded":"2024-04-26T20:02:44.189160+0000","last_fullsized":"2024-04-26T20:02:44.189160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:41.215693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.224982+0000","last_change":"2024-04-26T20:02:44.218725+0000","last_active":"2024-04-26T20:02:44.224982+0000","last_peered":"2024-04-26T20:02:44.224982+0000","last_clean":"2024-04-26T20:02:44.224982+0000","last_became_active":"2024-04-26T20:02:44.218390+0000","last_became_peered":"2024-04-26T20:02:44.218390+0000","last_unstale":"2024-04-26T20:02:44.224982+0000","last_undegraded":"2024-04-26T20:02:44.224982+0000","last_fullsized":"2024-04-26T20:02:44.224982+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:43.153211+0000","last_clean_scrub_stamp":"2024-04-26T20:02:43.153211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:16.073241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.237390+0000","last_change":"2024-04-26T20:02:43.190342+0000","last_active":"2024-04-26T20:02:44.237390+0000","last_peered":"2024-04-26T20:02:44.237390+0000","last_clean":"2024-04-26T20:02:44.237390+0000","last_became_active":"2024-04-26T20:02:43.190037+0000","last_became_peered":"2024-04-26T20:02:43.190037+0000","last_unstale":"2024-04-26T20:02:44.237390+0000","last_undegraded":"2024-04-26T20:02:44.237390+0000","last_fullsized":"2024-04-26T20:02:44.237390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:02:42.139194+0000","last_clean_scrub_stamp":"2024-04-26T20:02:42.139194+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:57.904098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":249,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T20:02:44.205280+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:02:44.205280+0000","last_peered":"2024-04-26T20:02:44.205280+0000","last_clean":"2024-04-26T20:02:44.205280+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:02:44.205280+0000","last_undegraded":"2024-04-26T20:02:44.205280+0000","last_fullsized":"2024-04-26T20:02:44.205280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28348,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343492,"statfs":{"total":96636764160,"available":96607735808,"internally_reserved":0,"allocated":1355776,"data_stored":912804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":761856,"data_stored":321912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":757760,"data_stored":321910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":757760,"data_stored":321890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":741376,"data_stored":315045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":323432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":324048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":912188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:02:49.139 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:02:49.139 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:02:49.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.388+0000 7fd967200640 1 Processor -- start 2024-04-26T20:02:49.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.388+0000 7fd967200640 1 -- start start 2024-04-26T20:02:49.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.389+0000 7fd967200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd968162850 0x7fd9680d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd967200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd9680d15d0 0x7fd9680d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd967200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 0x7fd9680b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd967200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fd9680d0a10 con 0x7fd968162850 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd967200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd9680d0cf0 con 0x7fd9680d15d0 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd967200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680b4a80 con 0x7fd9680d2c80 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd965e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd968162850 0x7fd9680d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd965e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fd968162850 0x7fd9680d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55926/0 (socket says 172.21.15.88:55926) 2024-04-26T20:02:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd965e00640 1 -- 172.21.15.88:0/661198175 learned_addr learned my addr 172.21.15.88:0/661198175 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd95d400640 1 --2- 172.21.15.88:0/661198175 >> v2:172.21.15.132:3300/0 conn(0x7fd9680d15d0 0x7fd9680d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd966800640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 0x7fd9680b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.390+0000 7fd965e00640 1 -- 172.21.15.88:0/661198175 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 msgr2=0x7fd9680b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.391+0000 7fd965e00640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 0x7fd9680b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.391+0000 7fd965e00640 1 -- 172.21.15.88:0/661198175 >> v2:172.21.15.132:3300/0 conn(0x7fd9680d15d0 msgr2=0x7fd9680d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.391+0000 7fd965e00640 1 --2- 172.21.15.88:0/661198175 >> v2:172.21.15.132:3300/0 conn(0x7fd9680d15d0 0x7fd9680d39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.391+0000 7fd965e00640 1 -- 172.21.15.88:0/661198175 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680b4d60 con 0x7fd968162850 2024-04-26T20:02:49.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.391+0000 7fd966800640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 0x7fd9680b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd965e00640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd9680d3480 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd96808d860 tx=0x7fd95802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1998ffa1b10a26bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd965400640 1 -- 172.21.15.88:0/661198175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958036070 con 0x7fd968162850 2024-04-26T20:02:49.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd965400640 1 -- 172.21.15.88:0/661198175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd958031600 con 0x7fd968162850 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd965400640 1 -- 172.21.15.88:0/661198175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9580319d0 con 0x7fd968162850 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd967200640 1 -- 172.21.15.88:0/661198175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 msgr2=0x7fd9680d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.392+0000 7fd967200640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd9680d3480 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd96808d860 tx=0x7fd95802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 -- 172.21.15.88:0/661198175 shutdown_connections 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d2c80 0x7fd9680b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 --2- 172.21.15.88:0/661198175 >> v2:172.21.15.132:3300/0 conn(0x7fd9680d15d0 0x7fd9680d39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 --2- 172.21.15.88:0/661198175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd9680d3480 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 -- 172.21.15.88:0/661198175 >> 172.21.15.88:0/661198175 conn(0x7fd9680d4f30 msgr2=0x7fd9680d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 -- 172.21.15.88:0/661198175 shutdown_connections 2024-04-26T20:02:49.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.393+0000 7fd967200640 1 -- 172.21.15.88:0/661198175 wait complete. 2024-04-26T20:02:49.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.394+0000 7fd967200640 1 Processor -- start 2024-04-26T20:02:49.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.395+0000 7fd967200640 1 -- start start 2024-04-26T20:02:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.395+0000 7fd967200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d15d0 0x7fd96815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd967200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9680d2c80 0x7fd9680cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd967200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd967200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9680d0740 con 0x7fd968162850 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd967200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd9680d05c0 con 0x7fd9680d2c80 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd967200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680b6d10 con 0x7fd9680d15d0 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55932/0 (socket says 172.21.15.88:55932) 2024-04-26T20:02:49.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 -- 172.21.15.88:0/3474578493 learned_addr learned my addr 172.21.15.88:0/3474578493 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd965e00640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d15d0 0x7fd96815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 -- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d15d0 msgr2=0x7fd96815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d15d0 0x7fd96815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 -- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9680d2c80 msgr2=0x7fd9680cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd95d400640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9680d2c80 0x7fd9680cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9680d2c80 0x7fd9680cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.396+0000 7fd966800640 1 -- 172.21.15.88:0/3474578493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680cd3d0 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd966800640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd96000c820 tx=0x7fd96000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960015070 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd960004240 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96000ce10 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9680cd6b0 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.397+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9680b8bc0 con 0x7fd968162850 2024-04-26T20:02:49.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.398+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd968050760 con 0x7fd968162850 2024-04-26T20:02:49.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.399+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd960007830 con 0x7fd968162850 2024-04-26T20:02:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.401+0000 7fd95ea00640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 0x7fd94406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.401+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fd96008bf20 con 0x7fd968162850 2024-04-26T20:02:49.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.401+0000 7fd965e00640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 0x7fd94406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.402+0000 7fd965e00640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 0x7fd94406ee40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd958033000 tx=0x7fd958009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.405+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd96005a250 con 0x7fd968162850 2024-04-26T20:02:49.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.679+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd9680d3260 con 0x7fd968162850 2024-04-26T20:02:49.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.683+0000 7fd95ea00640 1 -- 172.21.15.88:0/3474578493 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fd960059b50 con 0x7fd968162850 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBzByxmBYQlMhAAyXC23evUKZK39eXxzMWNSw== 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:02:49.685 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-04-26T20:02:49.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.686+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 msgr2=0x7fd94406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.686+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 0x7fd94406ee40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd958033000 tx=0x7fd958009000 comp rx=0 tx=0).stop 2024-04-26T20:02:49.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.686+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 msgr2=0x7fd96806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.686+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd96000c820 tx=0x7fd96000cce0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 shutdown_connections 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd968162850 0x7fd96806cdd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9680d2c80 0x7fd9680cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd94406c980 0x7fd94406ee40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 --2- 172.21.15.88:0/3474578493 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd9680d15d0 0x7fd96815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 >> 172.21.15.88:0/3474578493 conn(0x7fd9680d4f30 msgr2=0x7fd9680d5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 shutdown_connections 2024-04-26T20:02:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.687+0000 7fd967200640 1 -- 172.21.15.88:0/3474578493 wait complete. 2024-04-26T20:02:49.707 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:02:49.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.907+0000 7fa72e200640 1 Processor -- start 2024-04-26T20:02:49.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.907+0000 7fa72e200640 1 -- start start 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa72e200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa72806c770 0x7fa72806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa72e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa728073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa72e200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72806d090 con 0x7fa72815cb70 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa72e200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa7280af460 con 0x7fa72806c770 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa72e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280ae850 con 0x7fa72815cfd0 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa726a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa72815cb70 0x7fa728073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37402/0 (socket says 172.21.15.88:37402) 2024-04-26T20:02:49.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.909+0000 7fa726a00640 1 -- 172.21.15.88:0/2417203258 learned_addr learned my addr 172.21.15.88:0/2417203258 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243837360 0 0) 0x7fa72806d090 con 0x7fa72815cb70 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa727400640 1 --2- 172.21.15.88:0/2417203258 >> v2:172.21.15.132:3300/0 conn(0x7fa72806c770 0x7fa72806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa727e00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa728073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70c003520 con 0x7fa72815cb70 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393987003 0 0) 0x7fa70c003520 con 0x7fa72815cb70 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 msgr2=0x7fa728073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa728073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 >> v2:172.21.15.132:3300/0 conn(0x7fa72806c770 msgr2=0x7fa72806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 --2- 172.21.15.88:0/2417203258 >> v2:172.21.15.132:3300/0 conn(0x7fa72806c770 0x7fa72806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280aeb30 con 0x7fa72815cb70 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa714001670 con 0x7fa72815cb70 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 >> v1:172.21.15.88:6789/0 conn(0x7fa72815cb70 legacy=0x7fa728073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa70c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c003fe0 0x7fa70c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280af460 con 0x7fa70c003fe0 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280ae850 con 0x7fa70c004d40 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.910+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa70c00bac0 con 0x7fa72815cfd0 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c003fe0 0x7fa70c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa727e00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 -- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 msgr2=0x7fa70c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa727400640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa70c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72815cfd0 0x7fa70c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 -- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c004d40 msgr2=0x7fa70c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c004d40 0x7fa70c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.911+0000 7fa726a00640 1 -- 172.21.15.88:0/2417203258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c003520 con 0x7fa70c003fe0 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.912+0000 7fa726a00640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c003fe0 0x7fa70c004800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c210 tx=0x7fa70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325e3cf7c8713ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.912+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714007d50 con 0x7fa70c003fe0 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.912+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714005af0 con 0x7fa70c003fe0 2024-04-26T20:02:49.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.912+0000 7fa726000640 1 -- 172.21.15.88:0/2417203258 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714007070 con 0x7fa70c003fe0 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 -- 172.21.15.88:0/2417203258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c003fe0 msgr2=0x7fa70c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 --2- 172.21.15.88:0/2417203258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c003fe0 0x7fa70c004800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c210 tx=0x7fa70c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa727e00640 1 -- 172.21.15.88:0/2417203258 reap_dead start 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 -- 172.21.15.88:0/2417203258 shutdown_connections 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 -- 172.21.15.88:0/2417203258 >> 172.21.15.88:0/2417203258 conn(0x7fa7280b2d50 msgr2=0x7fa7280b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 -- 172.21.15.88:0/2417203258 shutdown_connections 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.913+0000 7fa72e200640 1 -- 172.21.15.88:0/2417203258 wait complete. 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.915+0000 7fa72e200640 1 Processor -- start 2024-04-26T20:02:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.915+0000 7fa72e200640 1 -- start start 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c003fe0 0x7fa728155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 0x7fa72809dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa72815c600 con 0x7fa72815ad10 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280af0f0 con 0x7fa70c003fe0 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa72e200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa72815c780 con 0x7fa72809db30 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa727e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa727e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55948/0 (socket says 172.21.15.88:55948) 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.916+0000 7fa727e00640 1 -- 172.21.15.88:0/2754656258 learned_addr learned my addr 172.21.15.88:0/2754656258 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727400640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c003fe0 0x7fa728155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa726a00640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 0x7fa72809dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 -- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 msgr2=0x7fa72809dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 0x7fa72809dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 -- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c003fe0 msgr2=0x7fa728155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c003fe0 0x7fa728155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 -- 172.21.15.88:0/2754656258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280cd9a0 con 0x7fa72815ad10 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa726a00640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 0x7fa72809dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa727e00640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa72806e300 tx=0x7fa720009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720007d20 con 0x7fa72815ad10 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa720019070 con 0x7fa72815ad10 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720014650 con 0x7fa72815ad10 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7280cdc80 con 0x7fa72815ad10 2024-04-26T20:02:49.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.917+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7280ce240 con 0x7fa72815ad10 2024-04-26T20:02:49.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.919+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa728069540 con 0x7fa72815ad10 2024-04-26T20:02:49.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.919+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa7200147f0 con 0x7fa72815ad10 2024-04-26T20:02:49.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.920+0000 7fa71f400640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 0x7fa70806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:49.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.920+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fa72008c220 con 0x7fa72815ad10 2024-04-26T20:02:49.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.921+0000 7fa727400640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 0x7fa70806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:49.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.921+0000 7fa727400640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 0x7fa70806ee10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7280b1e80 tx=0x7fa718008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:49.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:49.925+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa72005a450 con 0x7fa72815ad10 2024-04-26T20:02:50.141 INFO:tasks.ceph.mds.d.smithi132.stderr:2024-04-26T20:02:50.140+0000 7f1121e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:02:50.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.165+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa72806ec40 con 0x7fa72815ad10 2024-04-26T20:02:50.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.166+0000 7fa71f400640 1 -- 172.21.15.88:0/2754656258 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5186 (secure 0 0 0) 0x7fa720059d50 con 0x7fa72815ad10 2024-04-26T20:02:50.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:02:50.167 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:02:50.167 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 39 2024-04-26T20:02:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.168+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 msgr2=0x7fa70806ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.168+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 0x7fa70806ee10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7280b1e80 tx=0x7fa718008040 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.168+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 msgr2=0x7fa72815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.168+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa72806e300 tx=0x7fa720009d20 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 shutdown_connections 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa72815ad10 0x7fa72815b1c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa70806c950 0x7fa70806ee10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa72809db30 0x7fa72809dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 --2- 172.21.15.88:0/2754656258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa70c003fe0 0x7fa728155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 >> 172.21.15.88:0/2754656258 conn(0x7fa7280b2d50 msgr2=0x7fa7280b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 shutdown_connections 2024-04-26T20:02:50.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.169+0000 7fa72e200640 1 -- 172.21.15.88:0/2754656258 wait complete. 2024-04-26T20:02:50.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:02:43.647939+0000', 'modified': '2024-04-26T20:02:46.699316+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6156}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6156': {'gid': 6156, 'name': 'd', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6835/1355496095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6833', 'nonce': 1355496095}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 1355496095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6156, 'qdb_cluster': [6156]} 2024-04-26T20:02:50.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:02:50.592 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T20:02:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.802+0000 7f21eb200640 1 Processor -- start 2024-04-26T20:02:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.803+0000 7f21eb200640 1 -- start start 2024-04-26T20:02:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21eb200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21ec15f1b0 0x7f21ec0d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21eb200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0d3f50 con 0x7f21ec162b00 2024-04-26T20:02:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21eb200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0d36b0 con 0x7f21ec163650 2024-04-26T20:02:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21eb200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0d3990 con 0x7f21ec15f1b0 2024-04-26T20:02:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21e9e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21ec15f1b0 0x7f21ec0d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21e9400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f21ec162b00 0x7f21ec162f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37406/0 (socket says 172.21.15.88:37406) 2024-04-26T20:02:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.804+0000 7f21e9400640 1 -- 172.21.15.88:0/4216144944 learned_addr learned my addr 172.21.15.88:0/4216144944 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.805+0000 7f21e9e00640 1 -- 172.21.15.88:0/4216144944 >> v1:172.21.15.132:6789/0 conn(0x7f21ec163650 legacy=0x7f21ec0b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.805+0000 7f21e9e00640 1 -- 172.21.15.88:0/4216144944 >> v1:172.21.15.88:6789/0 conn(0x7f21ec162b00 legacy=0x7f21ec162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.805+0000 7f21e9e00640 1 -- 172.21.15.88:0/4216144944 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0d3c70 con 0x7f21ec15f1b0 2024-04-26T20:02:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.806+0000 7f21e9e00640 1 --2- 172.21.15.88:0/4216144944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0d5990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009d90 tx=0x7f21dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfa84f488755397 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.806+0000 7f21e8a00640 1 -- 172.21.15.88:0/4216144944 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc007740 con 0x7f21ec15f1b0 2024-04-26T20:02:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.806+0000 7f21e8a00640 1 -- 172.21.15.88:0/4216144944 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc0078e0 con 0x7f21ec15f1b0 2024-04-26T20:02:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.806+0000 7f21e8a00640 1 -- 172.21.15.88:0/4216144944 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc007cb0 con 0x7f21ec15f1b0 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 -- 172.21.15.88:0/4216144944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 msgr2=0x7f21ec0d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 --2- 172.21.15.88:0/4216144944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0d5990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009d90 tx=0x7f21dc02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 -- 172.21.15.88:0/4216144944 shutdown_connections 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 --2- 172.21.15.88:0/4216144944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0d5990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 -- 172.21.15.88:0/4216144944 >> 172.21.15.88:0/4216144944 conn(0x7f21ec09e230 msgr2=0x7f21ec0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 -- 172.21.15.88:0/4216144944 shutdown_connections 2024-04-26T20:02:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.807+0000 7f21eb200640 1 -- 172.21.15.88:0/4216144944 wait complete. 2024-04-26T20:02:50.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.809+0000 7f21eb200640 1 Processor -- start 2024-04-26T20:02:50.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.809+0000 7f21eb200640 1 -- start start 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21ec162b00 0x7f21ec0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec163650 0x7f21ec0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0b89c0 con 0x7f21ec15f1b0 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0d33a0 con 0x7f21ec163650 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21eb200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0b8b40 con 0x7f21ec162b00 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55952/0 (socket says 172.21.15.88:55952) 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.810+0000 7f21e9e00640 1 -- 172.21.15.88:0/1690580194 learned_addr learned my addr 172.21.15.88:0/1690580194 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:50.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9400640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21ec162b00 0x7f21ec0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21ea800640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec163650 0x7f21ec0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 -- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21ec162b00 msgr2=0x7f21ec0ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21ec162b00 0x7f21ec0ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 -- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec163650 msgr2=0x7f21ec0cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec163650 0x7f21ec0cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 -- 172.21.15.88:0/1690580194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0cae60 con 0x7f21ec15f1b0 2024-04-26T20:02:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.811+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009eb0 tx=0x7f21dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.812+0000 7f21e2000640 1 -- 172.21.15.88:0/1690580194 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc040070 con 0x7f21ec15f1b0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.812+0000 7f21e2000640 1 -- 172.21.15.88:0/1690580194 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc031d00 con 0x7f21ec15f1b0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.812+0000 7f21e2000640 1 -- 172.21.15.88:0/1690580194 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc045660 con 0x7f21ec15f1b0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.812+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec0cb140 con 0x7f21ec15f1b0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.812+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec150500 con 0x7f21ec15f1b0 2024-04-26T20:02:50.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.814+0000 7f21e2000640 1 -- 172.21.15.88:0/1690580194 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f21dc039040 con 0x7f21ec15f1b0 2024-04-26T20:02:50.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.816+0000 7f21e2000640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 0x7f21c806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.816+0000 7f21e2000640 1 -- 172.21.15.88:0/1690580194 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f21dc0ad890 con 0x7f21ec15f1b0 2024-04-26T20:02:50.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.816+0000 7f21e9400640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 0x7f21c806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.816+0000 7f21f39fc740 1 Processor -- start 2024-04-26T20:02:50.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.816+0000 7f21f39fc740 1 -- start start 2024-04-26T20:02:50.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21e9400640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 0x7f21c806ee40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0b6cb0 tx=0x7f21d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 0x55a3f77c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a3f7724600 con 0x55a3f77fce80 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a3f76c1100 con 0x55a3f77eb270 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.817+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a3f77205c0 con 0x55a3f77c07b0 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55968/0 (socket says 172.21.15.88:55968) 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 -- 172.21.15.88:0/2499252952 learned_addr learned my addr 172.21.15.88:0/2499252952 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21ea800640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 0x55a3f77c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 -- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 msgr2=0x55a3f77c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 0x55a3f77c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 -- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 msgr2=0x55a3f77eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 -- 172.21.15.88:0/2499252952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f74362d0 con 0x55a3f77fce80 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 0x55a3f77c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.818+0000 7f21e9400640 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005a60 tx=0x7f21d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c61a01a0927f10e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.819+0000 7f21da600640 1 -- 172.21.15.88:0/2499252952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400e500 con 0x55a3f77fce80 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.819+0000 7f21da600640 1 -- 172.21.15.88:0/2499252952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d400e6a0 con 0x55a3f77fce80 2024-04-26T20:02:50.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.819+0000 7f21da600640 1 -- 172.21.15.88:0/2499252952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d400ea50 con 0x55a3f77fce80 2024-04-26T20:02:50.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.819+0000 7f21f39fc740 1 -- 172.21.15.88:0/2499252952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 msgr2=0x55a3f77c0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.819+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f21d4005a60 tx=0x7f21d4011000 comp rx=0 tx=0).stop 2024-04-26T20:02:50.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 -- 172.21.15.88:0/2499252952 shutdown_connections 2024-04-26T20:02:50.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77c0b90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2499252952 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77c07b0 0x55a3f77c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 -- 172.21.15.88:0/2499252952 >> 172.21.15.88:0/2499252952 conn(0x55a3f7754d10 msgr2=0x55a3f7806270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 -- 172.21.15.88:0/2499252952 shutdown_connections 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 -- 172.21.15.88:0/2499252952 wait complete. 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.820+0000 7f21f39fc740 1 -- start start 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.821+0000 7f21f39fc740 1 Processor -- start 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.821+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77c07b0 0x55a3f77ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.821+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77eb270 0x55a3f77ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21f39fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a3f76abf70 con 0x55a3f77fce80 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a3f77e8dc0 con 0x55a3f77c07b0 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a3f7800780 con 0x55a3f77eb270 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21f39fc740 10 client.0 _ll_register_callbacks cb 0x55a3f771a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:02:50.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55972/0 (socket says 172.21.15.88:55972) 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 -- 172.21.15.88:0/1570217226 learned_addr learned my addr 172.21.15.88:0/1570217226 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77c07b0 0x55a3f77ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21e9400640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77eb270 0x55a3f77ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 -- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77eb270 msgr2=0x55a3f77ffc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77eb270 0x55a3f77ffc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 -- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77c07b0 msgr2=0x55a3f77ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77c07b0 0x55a3f77ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.822+0000 7f21ea800640 1 -- 172.21.15.88:0/1570217226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f75006b0 con 0x55a3f77fce80 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21ea800640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cf30 tx=0x7f21e400e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43daefd2a62d234 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4018020 con 0x55a3f77fce80 2024-04-26T20:02:50.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f21e4013450 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d400e6a0 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f21e4022050 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 10 client.6288 fetch_fsmap learned FSMap version 39 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55a3f77eeff0 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d400e6a0 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 39) ==== 3494+0+0 (secure 0 0 0) 0x7f21e4025020 con 0x55a3f77fce80 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 10 client.6288 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 10 client.6288 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.823+0000 7f21f39fc740 20 client.6288 populate_metadata read hostname 'smithi088' 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.824+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f77ee380 0x55a3f77ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.824+0000 7f21f39fc740 4 client.6288 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.824+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 --> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f7732210 con 0x55a3f77ee380 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.824+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f77ee380 0x55a3f77ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.825+0000 7f21e9e00640 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f77ee380 0x55a3f77ee760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade1a7afbc0c06c7 server_cookie=b08d6f7ae8f0ecd8 in_seq=0 out_seq=0 2024-04-26T20:02:50.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.825+0000 7f21cea00640 10 client.6288 ms_handle_connect on v2:172.21.15.132:6833/1355496095 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.827+0000 7f21cea00640 1 -- 172.21.15.88:0/1570217226 <== mds.0 v2:172.21.15.132:6833/1355496095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3f7732210 con 0x55a3f77ee380 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.827+0000 7f21cea00640 10 client.6288 handle_command_reply: tid=0 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.827+0000 7f21f39fc740 1 client.6288 shutdown 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 msgr2=0x55a3f77eb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f21e400cf30 tx=0x7f21e400e660 comp rx=0 tx=0).stop 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 shutdown_connections 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f77ee380 0x55a3f77ee760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77fce80 0x55a3f77eb650 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77eb270 0x55a3f77ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 --2- 172.21.15.88:0/1570217226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77c07b0 0x55a3f77ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21cea00640 0 client.6288 ms_handle_reset on v2:172.21.15.132:6833/1355496095 2024-04-26T20:02:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 >> 172.21.15.88:0/1570217226 conn(0x55a3f7754d10 msgr2=0x55a3f77e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 shutdown_connections 2024-04-26T20:02:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 1 -- 172.21.15.88:0/1570217226 wait complete. 2024-04-26T20:02:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.828+0000 7f21f39fc740 20 client.6288 trim_cache size 0 max 16384 2024-04-26T20:02:50.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.848+0000 7f21f39fc740 1 Processor -- start 2024-04-26T20:02:50.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.848+0000 7f21f39fc740 1 -- start start 2024-04-26T20:02:50.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.848+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f77ff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21f39fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec200810 con 0x55a3f77c07b0 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec151450 con 0x55a3f77eb270 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec151aa0 con 0x55a3f77fce80 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55980/0 (socket says 172.21.15.88:55980) 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 -- 172.21.15.88:0/2983814279 learned_addr learned my addr 172.21.15.88:0/2983814279 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 -- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 msgr2=0x55a3f77ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f77ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 -- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 msgr2=0x55a3f77e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 -- 172.21.15.88:0/2983814279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f78002b0 con 0x55a3f77c07b0 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.849+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e80 tx=0x7f21dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb65f389d71ad14e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21d9c00640 1 -- 172.21.15.88:0/2983814279 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc08f070 con 0x55a3f77c07b0 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21d9c00640 1 -- 172.21.15.88:0/2983814279 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc0ad040 con 0x55a3f77c07b0 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21d9c00640 1 -- 172.21.15.88:0/2983814279 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc04a040 con 0x55a3f77c07b0 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 -- 172.21.15.88:0/2983814279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 msgr2=0x55a3f77e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e80 tx=0x7f21dc009750 comp rx=0 tx=0).stop 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 -- 172.21.15.88:0/2983814279 shutdown_connections 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f77ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f77e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 --2- 172.21.15.88:0/2983814279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77e9690 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 -- 172.21.15.88:0/2983814279 >> 172.21.15.88:0/2983814279 conn(0x55a3f7754d10 msgr2=0x55a3f77b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.850+0000 7f21f39fc740 1 -- 172.21.15.88:0/2983814279 shutdown_connections 2024-04-26T20:02:50.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.851+0000 7f21f39fc740 1 -- 172.21.15.88:0/2983814279 wait complete. 2024-04-26T20:02:50.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.851+0000 7f21f39fc740 1 -- start start 2024-04-26T20:02:50.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.851+0000 7f21f39fc740 1 Processor -- start 2024-04-26T20:02:50.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f78838e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f7818030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a3f77e7ba0 con 0x55a3f77c07b0 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x55a3f77e80b0 con 0x55a3f77eb270 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x55a3f77ebdb0 con 0x55a3f77fce80 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.852+0000 7f21f39fc740 10 client.0 _ll_register_callbacks cb 0x7f21e4006e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55994/0 (socket says 172.21.15.88:55994) 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 -- 172.21.15.88:0/210201169 learned_addr learned my addr 172.21.15.88:0/210201169 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21ea800640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f7818030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9400640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f78838e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 -- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 msgr2=0x55a3f7818030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f7818030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 -- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 msgr2=0x55a3f78838e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f78838e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 -- 172.21.15.88:0/210201169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f7883e20 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21ea800640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f7818030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21e9e00640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e80 tx=0x7f21e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f69894c15c4377d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.853+0000 7f21cea00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc08f070 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f21dc077b50 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc0ad040 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f21dc0840c0 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 10 client.6294 fetch_fsmap learned FSMap version 39 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55a3f77fc780 con 0x55a3f77c07b0 2024-04-26T20:02:50.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc0ad040 con 0x55a3f77c07b0 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 39) ==== 3494+0+0 (secure 0 0 0) 0x7f21dc07b740 con 0x55a3f77c07b0 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 10 client.6294 fetch_fsmap finished waiting for FSMap version 39 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 10 client.6294 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 20 client.6294 populate_metadata read hostname 'smithi088' 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f78149d0 0x55a3f7814db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 4 client.6294 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.854+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 --> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] -- command(tid 0: {"prefix": "status"}) -- 0x55a3f772c080 con 0x55a3f78149d0 2024-04-26T20:02:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.855+0000 7f21e9400640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f78149d0 0x55a3f7814db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:50.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.856+0000 7f21e9400640 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f78149d0 0x55a3f7814db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57db43c699114a0 server_cookie=33e51dfabe92d416 in_seq=0 out_seq=0 2024-04-26T20:02:50.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.856+0000 7f21cea00640 10 client.6294 ms_handle_connect on v2:172.21.15.132:6833/1355496095 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21cea00640 1 -- 172.21.15.88:0/210201169 <== mds.0 v2:172.21.15.132:6833/1355496095 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a3f772c080 con 0x55a3f78149d0 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21cea00640 10 client.6294 handle_command_reply: tid=0 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 client.6294 shutdown 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 msgr2=0x55a3f77ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009e80 tx=0x7f21e400c850 comp rx=0 tx=0).stop 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 shutdown_connections 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:6833/1355496095,v1:172.21.15.132:6835/1355496095] conn(0x55a3f78149d0 0x55a3f7814db0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x55a3f77fce80 0x55a3f7818030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21cea00640 0 client.6294 ms_handle_reset on v2:172.21.15.132:6833/1355496095 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a3f77eb270 0x55a3f78838e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 --2- 172.21.15.88:0/210201169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a3f77c07b0 0x55a3f77ba6d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 >> 172.21.15.88:0/210201169 conn(0x55a3f7754d10 msgr2=0x55a3f77c2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 shutdown_connections 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.857+0000 7f21f39fc740 1 -- 172.21.15.88:0/210201169 wait complete. 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21f39fc740 20 client.6294 trim_cache size 0 max 16384 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "100f2996-174e-4373-bd56-473fe683ba7a", 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6156, 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6525598969999997, 2024-04-26T20:02:50.861 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 39, 2024-04-26T20:02:50.862 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 63, 2024-04-26T20:02:50.862 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 63, 2024-04-26T20:02:50.862 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 12.117150298 2024-04-26T20:02:50.862 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:50.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 msgr2=0x7f21c806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 0x7f21c806ee40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0b6cb0 tx=0x7f21d4009040 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 msgr2=0x7f21ec0a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009eb0 tx=0x7f21dc004a40 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 shutdown_connections 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec163650 0x7f21ec0cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806c980 0x7f21c806ee40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21ec162b00 0x7f21ec0ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 --2- 172.21.15.88:0/1690580194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec15f1b0 0x7f21ec0a2530 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 >> 172.21.15.88:0/1690580194 conn(0x7f21ec09e230 msgr2=0x7f21ec0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 shutdown_connections 2024-04-26T20:02:50.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:50.858+0000 7f21eb200640 1 -- 172.21.15.88:0/1690580194 wait complete. 2024-04-26T20:02:50.877 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:02:51.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.079+0000 7fc35a600640 1 Processor -- start 2024-04-26T20:02:51.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.079+0000 7fc35a600640 1 -- start start 2024-04-26T20:02:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc35a600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fc35406c770 0x7fc35406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc35a600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35415cb70 0x7fc354073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc35a600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35406d090 con 0x7fc35415cfd0 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc35a600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc3540af460 con 0x7fc35415cb70 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc35a600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fc3540ae850 con 0x7fc35406c770 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35415cb70 0x7fc354073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc35415cb70 0x7fc354073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56006/0 (socket says 172.21.15.88:56006) 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 -- 172.21.15.88:0/1022038115 learned_addr learned my addr 172.21.15.88:0/1022038115 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 -- 172.21.15.88:0/1022038115 >> v2:172.21.15.132:3300/0 conn(0x7fc35406c770 msgr2=0x7fc35406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 --2- 172.21.15.88:0/1022038115 >> v2:172.21.15.132:3300/0 conn(0x7fc35406c770 0x7fc35406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.081+0000 7fc352a00640 1 -- 172.21.15.88:0/1022038115 >> v1:172.21.15.132:6789/0 conn(0x7fc35415cfd0 legacy=0x7fc354073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.082+0000 7fc352a00640 1 -- 172.21.15.88:0/1022038115 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3540aeb30 con 0x7fc35415cb70 2024-04-26T20:02:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.082+0000 7fc352a00640 1 --2- 172.21.15.88:0/1022038115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc354073120 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc3540741f0 tx=0x7fc33c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328f99ff1950b7df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:51.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc352000640 1 -- 172.21.15.88:0/1022038115 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c037070 con 0x7fc35415cb70 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc352000640 1 -- 172.21.15.88:0/1022038115 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c032440 con 0x7fc35415cb70 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc352000640 1 -- 172.21.15.88:0/1022038115 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c032810 con 0x7fc35415cb70 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 -- 172.21.15.88:0/1022038115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 msgr2=0x7fc354073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 --2- 172.21.15.88:0/1022038115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc354073120 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc3540741f0 tx=0x7fc33c02a230 comp rx=0 tx=0).stop 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 -- 172.21.15.88:0/1022038115 shutdown_connections 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 --2- 172.21.15.88:0/1022038115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc354073120 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 --2- 172.21.15.88:0/1022038115 >> v2:172.21.15.132:3300/0 conn(0x7fc35406c770 0x7fc35406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.083+0000 7fc35a600640 1 -- 172.21.15.88:0/1022038115 >> 172.21.15.88:0/1022038115 conn(0x7fc3540b2d50 msgr2=0x7fc3540b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:51.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.084+0000 7fc35a600640 1 -- 172.21.15.88:0/1022038115 shutdown_connections 2024-04-26T20:02:51.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.084+0000 7fc35a600640 1 -- 172.21.15.88:0/1022038115 wait complete. 2024-04-26T20:02:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.085+0000 7fc35a600640 1 Processor -- start 2024-04-26T20:02:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.085+0000 7fc35a600640 1 -- start start 2024-04-26T20:02:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35406c770 0x7fc354152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35415cfd0 0x7fc3540b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc35415c600 con 0x7fc35415cb70 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc3540af0f0 con 0x7fc35415cfd0 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.086+0000 7fc35a600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc35415c780 con 0x7fc35406c770 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56022/0 (socket says 172.21.15.88:56022) 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 -- 172.21.15.88:0/3487784459 learned_addr learned my addr 172.21.15.88:0/3487784459 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 -- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35406c770 msgr2=0x7fc354152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35406c770 0x7fc354152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 -- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35415cfd0 msgr2=0x7fc3540b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35415cfd0 0x7fc3540b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 -- 172.21.15.88:0/3487784459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3541e6740 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.087+0000 7fc352a00640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02a270 tx=0x7fc33c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.088+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c041070 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.088+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c03c870 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.088+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c037040 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.088+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3541e6a20 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.088+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3541e6fe0 con 0x7fc35415cb70 2024-04-26T20:02:51.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.089+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc354069540 con 0x7fc35415cb70 2024-04-26T20:02:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.090+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc33c044070 con 0x7fc35415cb70 2024-04-26T20:02:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.091+0000 7fc34b400640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 0x7fc33006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.092+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fc33c0ac820 con 0x7fc35415cb70 2024-04-26T20:02:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.092+0000 7fc353400640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 0x7fc33006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:51.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.092+0000 7fc353400640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 0x7fc33006ee40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc3540d0870 tx=0x7fc344009250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:51.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.096+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc33c07abf0 con 0x7fc35415cb70 2024-04-26T20:02:51.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.345+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc354068a90 con 0x7fc35415cb70 2024-04-26T20:02:51.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.346+0000 7fc34b400640 1 -- 172.21.15.88:0/3487784459 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) ==== 53+0+73 (secure 0 0 0) 0x7fc33c07a4f0 con 0x7fc35415cb70 2024-04-26T20:02:51.346 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 msgr2=0x7fc33006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 0x7fc33006ee40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc3540d0870 tx=0x7fc344009250 comp rx=0 tx=0).stop 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 msgr2=0x7fc3540b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc33c02a270 tx=0x7fc33c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 shutdown_connections 2024-04-26T20:02:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc35415cfd0 0x7fc3540b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc35415cb70 0x7fc3540b18f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc33006c980 0x7fc33006ee40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 --2- 172.21.15.88:0/3487784459 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc35406c770 0x7fc354152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 >> 172.21.15.88:0/3487784459 conn(0x7fc3540b2d50 msgr2=0x7fc3540b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 shutdown_connections 2024-04-26T20:02:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51.348+0000 7fc35a600640 1 -- 172.21.15.88:0/3487784459 wait complete. 2024-04-26T20:02:51.367 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:02:51.367 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:02:51.368 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:51.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:51.368 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-04-26T20:02:51.417 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:02:51.417 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:02:51.417 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:02:51.417 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:51.418 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42405sec preferred_lft 42405sec 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.419 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::9c6e:acff:fec3:49a/64 scope link 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether c6:33:4d:d7:93:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::c433:4dff:fed7:93b6/64 scope link 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 2e:af:04:1f:b3:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::2caf:4ff:fe1f:b3fa/64 scope link 2024-04-26T20:02:51.420 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:51.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:51.421 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:02:51.496 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-04-26T20:02:51.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:51.500 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-04-26T20:02:51.553 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-04-26T20:02:51.553 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:02:51.553 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:02:51.553 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:02:51.553 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:02:51.554 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-04-26T20:02:51.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:02:51.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:51.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:51.762 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:02:51.810 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:02:51.810 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:02:51.811 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:51.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:51.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-04-26T20:02:51.892 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:02:51.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:51.892 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:51.946 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:02:51.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:51.947 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:51.996 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:02:52.081 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:02:52.137 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:52.138 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:52.139 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:52.140 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:52.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:52.141 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:02:52.192 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:02:52.193 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:02:52.200 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-04-26T20:02:52.274 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-04-26T20:02:52.314 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:52.315 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:52.316 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:02:52.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:52.317 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-04-26T20:02:52.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2024-04-26T20:02:52.366+0000 7f3640d3b480 -1 init, newargv = 0x5610c7a983e0 newargc=15 2024-04-26T20:02:52.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[53646]: starting ceph client 2024-04-26T20:02:52.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[53646]: starting fuse 2024-04-26T20:02:52.400 INFO:teuthology.orchestra.run.smithi088.stdout:41 2024-04-26T20:02:52.400 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:02:52.401 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:02:52.402 DEBUG:teuthology.orchestra.run.smithi088:> 2024-04-26T20:02:52.403 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2024-04-26T20:02:52.403 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-04-26T20:02:52.590 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.53646.asok 2024-04-26T20:02:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:52.606 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53646.asok 2024-04-26T20:02:52.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:52.606 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53646.asok status 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "53646", 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-04-26T20:02:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6306, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 6306 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 812453214 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-04-26T20:02:52.832 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 812453214 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.6306 192.168.0.1:0/812453214", 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/812453214", 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 39, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 63, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-04-26T20:02:52.833 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:02:52.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:52.847 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-04-26T20:02:52.847 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:52.900 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-04-26T20:02:52.901 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:52.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:52.901 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:02:52.984 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:02:53.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.185+0000 7f73c1e00640 1 Processor -- start 2024-04-26T20:02:53.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.185+0000 7f73c1e00640 1 -- start start 2024-04-26T20:02:53.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.186+0000 7f73c1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73c1e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f73bc06fb50 0x7f73bc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73c1e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f73bc06ff70 0x7f73bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73c1e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f73bc0ae370 con 0x7f73bc06ff70 2024-04-26T20:02:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73c1e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f73bc0ae5f0 con 0x7f73bc06fb50 2024-04-26T20:02:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73c1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc072010 con 0x7f73bc06e130 2024-04-26T20:02:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f73bc06ff70 0x7f73bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f73bc06ff70 0x7f73bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56034/0 (socket says 172.21.15.88:56034) 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 -- 172.21.15.88:0/2725931705 learned_addr learned my addr 172.21.15.88:0/2725931705 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73baa00640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73b2000640 1 --2- 172.21.15.88:0/2725931705 >> v2:172.21.15.132:3300/0 conn(0x7f73bc06fb50 0x7f73bc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 -- 172.21.15.88:0/2725931705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 msgr2=0x7f73bc0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 -- 172.21.15.88:0/2725931705 >> v2:172.21.15.132:3300/0 conn(0x7f73bc06fb50 msgr2=0x7f73bc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.187+0000 7f73bb400640 1 --2- 172.21.15.88:0/2725931705 >> v2:172.21.15.132:3300/0 conn(0x7f73bc06fb50 0x7f73bc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.188+0000 7f73bb400640 1 -- 172.21.15.88:0/2725931705 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc0722f0 con 0x7f73bc06ff70 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73bb400640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06ff70 0x7f73bc0735f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0b3980 tx=0x7f73b402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b48093a3fb0a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73ba000640 1 -- 172.21.15.88:0/2725931705 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b4036070 con 0x7f73bc06ff70 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73ba000640 1 -- 172.21.15.88:0/2725931705 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b4031600 con 0x7f73bc06ff70 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73ba000640 1 -- 172.21.15.88:0/2725931705 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b40319d0 con 0x7f73bc06ff70 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73c1e00640 1 -- 172.21.15.88:0/2725931705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06ff70 msgr2=0x7f73bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.189+0000 7f73c1e00640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06ff70 0x7f73bc0735f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0b3980 tx=0x7f73b402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 -- 172.21.15.88:0/2725931705 shutdown_connections 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06ff70 0x7f73bc0735f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 --2- 172.21.15.88:0/2725931705 >> v2:172.21.15.132:3300/0 conn(0x7f73bc06fb50 0x7f73bc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 --2- 172.21.15.88:0/2725931705 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 -- 172.21.15.88:0/2725931705 >> 172.21.15.88:0/2725931705 conn(0x7f73bc0b2d50 msgr2=0x7f73bc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 -- 172.21.15.88:0/2725931705 shutdown_connections 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.190+0000 7f73c1e00640 1 -- 172.21.15.88:0/2725931705 wait complete. 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.191+0000 7f73c1e00640 1 Processor -- start 2024-04-26T20:02:53.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.191+0000 7f73c1e00640 1 -- start start 2024-04-26T20:02:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 0x7f73bc155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc072b90 con 0x7f73bc06fb50 2024-04-26T20:02:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc0728b0 con 0x7f73bc06ff70 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.192+0000 7f73c1e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc0b4810 con 0x7f73bc06e130 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56036/0 (socket says 172.21.15.88:56036) 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 -- 172.21.15.88:0/3660491824 learned_addr learned my addr 172.21.15.88:0/3660491824 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73bb400640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 0x7f73bc155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 -- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 msgr2=0x7f73bc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 -- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 msgr2=0x7f73bc155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 0x7f73bc155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 -- 172.21.15.88:0/3660491824 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc156760 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73bb400640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 0x7f73bc155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.193+0000 7f73b2000640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73a400d830 tx=0x7f73a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.194+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400b6c0 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.194+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a4019070 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.194+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc156a40 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.194+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a400ba80 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.194+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc157000 con 0x7f73bc06fb50 2024-04-26T20:02:53.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.195+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73bc069540 con 0x7f73bc06fb50 2024-04-26T20:02:53.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.196+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f73a401e460 con 0x7f73bc06fb50 2024-04-26T20:02:53.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.197+0000 7f73b3e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 0x7f739806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:53.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.197+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f73a408c390 con 0x7f73bc06fb50 2024-04-26T20:02:53.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.197+0000 7f73baa00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 0x7f739806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:53.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.198+0000 7f73baa00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 0x7f739806ee40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b7f0 tx=0x7f73ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:53.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.202+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73a405a6d0 con 0x7f73bc06fb50 2024-04-26T20:02:53.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.444+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73bc0b3ef0 con 0x7f73bc06fb50 2024-04-26T20:02:53.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.445+0000 7f73b3e00640 1 -- 172.21.15.88:0/3660491824 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) ==== 53+0+73 (secure 0 0 0) 0x7f73a4059fd0 con 0x7f73bc06fb50 2024-04-26T20:02:53.445 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 msgr2=0x7f739806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 0x7f739806ee40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b7f0 tx=0x7f73ac008040 comp rx=0 tx=0).stop 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 msgr2=0x7f73bc14cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73a400d830 tx=0x7f73a400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 shutdown_connections 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73bc06ff70 0x7f73bc155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73bc06fb50 0x7f73bc14cbf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f739806c980 0x7f739806ee40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 --2- 172.21.15.88:0/3660491824 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f73bc06e130 0x7f73bc0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:53.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 >> 172.21.15.88:0/3660491824 conn(0x7f73bc0b2d50 msgr2=0x7f73bc0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:53.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 shutdown_connections 2024-04-26T20:02:53.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:53.447+0000 7f73c1e00640 1 -- 172.21.15.88:0/3660491824 wait complete. 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:02:53.468 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:53.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:53.468 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-04-26T20:02:53.526 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:53.527 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42418sec preferred_lft 42418sec 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ccda:b9ff:fed8:349c/64 scope link 2024-04-26T20:02:53.528 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 82:29:7d:56:31:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::8029:7dff:fe56:31fa/64 scope link 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 92:6d:d7:ca:c5:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::906d:d7ff:feca:c5f7/64 scope link 2024-04-26T20:02:53.529 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:02:53.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:53.530 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:02:53.611 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-04-26T20:02:53.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:53.615 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-04-26T20:02:53.666 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-04-26T20:02:53.666 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:02:53.666 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:02:53.666 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:02:53.666 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:02:53.667 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:02:53.668 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-04-26T20:02:53.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:02:53.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:02:53.668 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:02:53.668 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-04-26T20:02:53.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:02:53.863 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:53.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:53.873 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:02:53.922 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:02:53.922 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:02:53.923 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:53.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:53.923 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-04-26T20:02:53.995 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:02:53.995 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:53.996 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:54.048 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:02:54.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:02:54.048 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:54.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:02:54.184 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:54.241 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:02:54.241 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:02:54.241 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:02:54.242 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:02:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:02:54.244 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:02:54.244 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:54.244 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:54.244 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:54.244 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:54.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:54.245 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:02:54.296 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:02:54.296 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:02:54.301 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-04-26T20:02:54.367 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:02:54.410 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-04-26T20:02:54.410 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-04-26T20:02:54.410 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-04-26T20:02:54.411 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-04-26T20:02:54.411 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:02:54.412 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:02:54.413 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:02:54.414 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-04-26T20:02:54.415 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:54.415 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:02:54.415 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:54.415 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:02:54.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:54.417 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-04-26T20:02:54.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:2024-04-26T20:02:54.463+0000 7fa403416480 -1 init, newargv = 0x555ffc7b18b0 newargc=15 2024-04-26T20:02:54.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[41337]: starting ceph client 2024-04-26T20:02:54.474 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[41337]: starting fuse 2024-04-26T20:02:54.497 INFO:teuthology.orchestra.run.smithi132.stdout:42 2024-04-26T20:02:54.497 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-04-26T20:02:54.498 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-04-26T20:02:54.499 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-04-26T20:02:54.500 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:02:54.500 DEBUG:teuthology.orchestra.run.smithi132:> 2024-04-26T20:02:54.500 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.1")) 2024-04-26T20:02:54.500 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-04-26T20:02:54.687 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.1.41337.asok 2024-04-26T20:02:54.690 INFO:teuthology.orchestra.run.smithi132.stderr:2024-04-26T20:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:02:54.703 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41337.asok 2024-04-26T20:02:54.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:54.704 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41337.asok status 2024-04-26T20:02:54.929 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-04-26T20:02:54.929 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "1", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "41337", 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 4970, 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-04-26T20:02:54.930 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 4970 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3622490194 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 3622490194 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-04-26T20:02:54.931 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.4970 192.168.0.1:0/3622490194", 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.1:0/3622490194", 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 39, 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 63, 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-04-26T20:02:54.932 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-04-26T20:02:54.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:54.944 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-04-26T20:02:54.944 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:54.996 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-04-26T20:02:54.997 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:54.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:54.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:02:55.077 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T20:02:55.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.291+0000 7f3d37200640 1 Processor -- start 2024-04-26T20:02:55.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.291+0000 7f3d37200640 1 -- start start 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d37200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3d380d0510 0x7f3d380cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d37200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d38160070 con 0x7f3d380c5b30 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d37200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d38160350 con 0x7f3d3806ed80 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d37200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3806e8f0 con 0x7f3d380d0510 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d35e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3d380c5b30 0x7f3d380c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37416/0 (socket says 172.21.15.88:37416) 2024-04-26T20:02:55.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.293+0000 7f3d35e00640 1 -- 172.21.15.88:0/1939498439 learned_addr learned my addr 172.21.15.88:0/1939498439 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120489053 0 0) 0x7f3d38160070 con 0x7f3d380c5b30 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d1c003520 con 0x7f3d380c5b30 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/1939498439 >> v2:172.21.15.132:3300/0 conn(0x7f3d380d0510 0x7f3d380cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1050204258 0 0) 0x7f3d1c003520 con 0x7f3d380c5b30 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 >> v2:172.21.15.132:3300/0 conn(0x7f3d380d0510 msgr2=0x7f3d380cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 --2- 172.21.15.88:0/1939498439 >> v2:172.21.15.132:3300/0 conn(0x7f3d380d0510 0x7f3d380cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 >> v1:172.21.15.132:6789/0 conn(0x7f3d3806ed80 legacy=0x7f3d380ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3806e2a0 con 0x7f3d380c5b30 2024-04-26T20:02:55.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f3d28001670 con 0x7f3d380c5b30 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 >> v1:172.21.15.88:6789/0 conn(0x7f3d380c5b30 legacy=0x7f3d380c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.294+0000 7f3d35400640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c003a00 0x7f3d1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35400640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d1c004800 0x7f3d1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35400640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3806e8f0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c00c200 con 0x7f3d1c003a00 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c00c400 con 0x7f3d1c004800 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d35e00640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c003a00 0x7f3d1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d1c004800 0x7f3d1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 -- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d1c004800 msgr2=0x7f3d1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d1c004800 0x7f3d1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 -- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c003a00 msgr2=0x7f3d1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:55.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d1c003a00 0x7f3d1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.295+0000 7f3d36800640 1 -- 172.21.15.88:0/1939498439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c003520 con 0x7f3d1c0052e0 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.296+0000 7f3d36800640 1 -- 172.21.15.88:0/1939498439 reap_dead start 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.296+0000 7f3d36800640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d1c00b9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3d30001e50 tx=0x7f3d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5298e7342c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30001a70 con 0x7f3d1c0052e0 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d30003970 con 0x7f3d1c0052e0 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d35400640 1 -- 172.21.15.88:0/1939498439 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30003d60 con 0x7f3d1c0052e0 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d37200640 1 -- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 msgr2=0x7f3d1c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d37200640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d1c00b9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3d30001e50 tx=0x7f3d1c005830 comp rx=0 tx=0).stop 2024-04-26T20:02:55.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d37200640 1 -- 172.21.15.88:0/1939498439 shutdown_connections 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d37200640 1 --2- 172.21.15.88:0/1939498439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d1c00b9c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.297+0000 7f3d37200640 1 -- 172.21.15.88:0/1939498439 >> 172.21.15.88:0/1939498439 conn(0x7f3d380d0a80 msgr2=0x7f3d380d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.298+0000 7f3d37200640 1 -- 172.21.15.88:0/1939498439 shutdown_connections 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.298+0000 7f3d37200640 1 -- 172.21.15.88:0/1939498439 wait complete. 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.299+0000 7f3d37200640 1 Processor -- start 2024-04-26T20:02:55.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 -- start start 2024-04-26T20:02:55.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 0x7f3d38072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d380b3550 0x7f3d380b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3806f1f0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3815f240 con 0x7f3d38072b40 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.300+0000 7f3d37200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3806f370 con 0x7f3d380b3550 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56068/0 (socket says 172.21.15.88:56068) 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 -- 172.21.15.88:0/732948587 learned_addr learned my addr 172.21.15.88:0/732948587 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d36800640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d380b3550 0x7f3d380b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 0x7f3d38072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 -- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d380b3550 msgr2=0x7f3d380b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d380b3550 0x7f3d380b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 -- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 msgr2=0x7f3d38072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:55.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 0x7f3d38072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d35e00640 1 -- 172.21.15.88:0/732948587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380b4280 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.301+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 0x7f3d38072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d35e00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e790 tx=0x7f3d2800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800c660 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d28018070 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800ca40 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d380b4560 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.302+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d380b4b20 con 0x7f3d1c0052e0 2024-04-26T20:02:55.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.303+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d380cced0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.304+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3d2800cbe0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.305+0000 7f3d2e000640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 0x7f3d1406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:55.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.305+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f3d2808b6a0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.305+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 0x7f3d1406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:55.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.306+0000 7f3d2fe00640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 0x7f3d1406ee70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d38071db0 tx=0x7f3d20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:55.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.310+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d380cced0 con 0x7f3d1c0052e0 2024-04-26T20:02:55.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:55.552+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3d380693c0 con 0x7f3d1c0052e0 2024-04-26T20:02:56.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.293+0000 7f3d2e000640 1 -- 172.21.15.88:0/732948587 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v40) ==== 136+0+0 (secure 0 0 0) 0x7f3d280599e0 con 0x7f3d1c0052e0 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 msgr2=0x7f3d1406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 0x7f3d1406ee70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d38071db0 tx=0x7f3d20008040 comp rx=0 tx=0).stop 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 msgr2=0x7f3d381532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e790 tx=0x7f3d2800ec50 comp rx=0 tx=0).stop 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 shutdown_connections 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3d1406c9b0 0x7f3d1406ee70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3d380b3550 0x7f3d380b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d38072b40 0x7f3d38072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 --2- 172.21.15.88:0/732948587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d1c0052e0 0x7f3d381532f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 >> 172.21.15.88:0/732948587 conn(0x7f3d380d0a80 msgr2=0x7f3d380ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 shutdown_connections 2024-04-26T20:02:56.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.296+0000 7f3d37200640 1 -- 172.21.15.88:0/732948587 wait complete. 2024-04-26T20:02:56.317 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T20:02:56.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.516+0000 7f8f42400640 1 Processor -- start 2024-04-26T20:02:56.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.517+0000 7f8f42400640 1 -- start start 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.518+0000 7f8f42400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f8f3c15cdf0 0x7f8f3c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.518+0000 7f8f42400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f3c0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.518+0000 7f8f42400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f3c0af9a0 con 0x7f8f3c0cd650 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.518+0000 7f8f42400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f8f3c0ae2c0 con 0x7f8f3c15cdf0 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.518+0000 7f8f42400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b05b0 con 0x7f8f3c06ce80 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f32a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8f3c0cd650 0x7f8f3c06c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:37420/0 (socket says 172.21.15.88:37420) 2024-04-26T20:02:56.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f32a00640 1 -- 172.21.15.88:0/2515670558 learned_addr learned my addr 172.21.15.88:0/2515670558 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:02:56.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f3c0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2515670558 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c15cdf0 0x7f8f3c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 517687133 0 0) 0x7f8f3c0af9a0 con 0x7f8f3c0cd650 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f20003520 con 0x7f8f3c0cd650 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.519+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3302234308 0 0) 0x7f8f20003520 con 0x7f8f3c0cd650 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 msgr2=0x7f8f3c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f3c0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c15cdf0 msgr2=0x7f8f3c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2515670558 >> v2:172.21.15.132:3300/0 conn(0x7f8f3c15cdf0 0x7f8f3c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c0afc80 con 0x7f8f3c0cd650 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f8f24001670 con 0x7f8f3c0cd650 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 >> v1:172.21.15.88:6789/0 conn(0x7f8f3c0cd650 legacy=0x7f8f3c06c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b05b0 con 0x7f8f200039b0 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2000ba20 con 0x7f8f20004800 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2000bc20 con 0x7f8f3c06ce80 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f32a00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.520+0000 7f8f3b400640 1 -- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 msgr2=0x7f8f200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f3b400640 1 -- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 msgr2=0x7f8f20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f3b400640 1 -- 172.21.15.88:0/2515670558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f20003520 con 0x7f8f200039b0 2024-04-26T20:02:56.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f3be00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c06ce80 0x7f8f200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:56.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.521+0000 7f8f32a00640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f20004800 0x7f8f20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.522+0000 7f8f3b400640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba00 tx=0x7f8f20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30996ed51011bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.522+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c030070 con 0x7f8f200039b0 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.522+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c02adb0 con 0x7f8f200039b0 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.522+0000 7f8f3aa00640 1 -- 172.21.15.88:0/2515670558 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00b600 con 0x7f8f200039b0 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 -- 172.21.15.88:0/2515670558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 msgr2=0x7f8f20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 --2- 172.21.15.88:0/2515670558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f20003dd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00ba00 tx=0x7f8f20005830 comp rx=0 tx=0).stop 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f3be00640 1 -- 172.21.15.88:0/2515670558 reap_dead start 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 -- 172.21.15.88:0/2515670558 shutdown_connections 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 -- 172.21.15.88:0/2515670558 >> 172.21.15.88:0/2515670558 conn(0x7f8f3c0b2d50 msgr2=0x7f8f3c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 -- 172.21.15.88:0/2515670558 shutdown_connections 2024-04-26T20:02:56.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.523+0000 7f8f42400640 1 -- 172.21.15.88:0/2515670558 wait complete. 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.525+0000 7f8f42400640 1 Processor -- start 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.525+0000 7f8f42400640 1 -- start start 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15ace0 0x7f8f3c09bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c1f8ab0 0x7f8f3c06f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c15d1d0 con 0x7f8f200039b0 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c0b0240 con 0x7f8f3c1f8ab0 2024-04-26T20:02:56.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f42400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c15d350 con 0x7f8f3c15ace0 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56082/0 (socket says 172.21.15.88:56082) 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 -- 172.21.15.88:0/1211933710 learned_addr learned my addr 172.21.15.88:0/1211933710 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 -- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15ace0 msgr2=0x7f8f3c09bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f32a00640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15ace0 0x7f8f3c09bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15ace0 0x7f8f3c09bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3be00640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c1f8ab0 0x7f8f3c06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 -- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c1f8ab0 msgr2=0x7f8f3c06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c1f8ab0 0x7f8f3c06f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.526+0000 7f8f3b400640 1 -- 172.21.15.88:0/1211933710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c15b450 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f3b400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bb20 tx=0x7f8f2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c043020 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f2c03f680 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00b600 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c15b730 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.527+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c15b160 con 0x7f8f200039b0 2024-04-26T20:02:56.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.528+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c069540 con 0x7f8f200039b0 2024-04-26T20:02:56.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.529+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8f2c046020 con 0x7f8f200039b0 2024-04-26T20:02:56.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.530+0000 7f8f33e00640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 0x7f8f1806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.530+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8f2c0ac770 con 0x7f8f200039b0 2024-04-26T20:02:56.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.530+0000 7f8f32a00640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 0x7f8f1806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.531+0000 7f8f32a00640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 0x7f8f1806ee70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f240017e0 tx=0x7f8f24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.535+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f2c04d030 con 0x7f8f200039b0 2024-04-26T20:02:56.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.760+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8f3c0add80 con 0x7f8f1806c9b0 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.762+0000 7f8f33e00640 1 -- 172.21.15.88:0/1211933710 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8f3c0add80 con 0x7f8f1806c9b0 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-26T20:02:56.763 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 msgr2=0x7f8f1806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 0x7f8f1806ee70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f240017e0 tx=0x7f8f24009040 comp rx=0 tx=0).stop 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 msgr2=0x7f8f3c15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00bb20 tx=0x7f8f2c004730 comp rx=0 tx=0).stop 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 shutdown_connections 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3c1f8ab0 0x7f8f3c06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8f1806c9b0 0x7f8f1806ee70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8f3c15ace0 0x7f8f3c09bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 --2- 172.21.15.88:0/1211933710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f200039b0 0x7f8f3c15dd50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 >> 172.21.15.88:0/1211933710 conn(0x7f8f3c0b2d50 msgr2=0x7f8f3c0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:56.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 shutdown_connections 2024-04-26T20:02:56.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.765+0000 7f8f42400640 1 -- 172.21.15.88:0/1211933710 wait complete. 2024-04-26T20:02:56.785 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:02:56.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.982+0000 7f8a13200640 1 Processor -- start 2024-04-26T20:02:56.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.982+0000 7f8a13200640 1 -- start start 2024-04-26T20:02:56.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a13200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a140b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a13200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8a14146280 0x7f8a141466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a13200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a140b4d20 con 0x7f8a14161140 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a13200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8a1406e5f0 con 0x7f8a14146280 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a13200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a14146c30 con 0x7f8a14160d60 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a12800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8a14146280 0x7f8a141466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a12800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8a14146280 0x7f8a141466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56086/0 (socket says 172.21.15.88:56086) 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a12800640 1 -- 172.21.15.88:0/4186083300 learned_addr learned my addr 172.21.15.88:0/4186083300 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.984+0000 7f8a11e00640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a140b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.985+0000 7f8a12800640 1 -- 172.21.15.88:0/4186083300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 msgr2=0x7f8a140b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.985+0000 7f8a12800640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a140b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.985+0000 7f8a12800640 1 -- 172.21.15.88:0/4186083300 >> v1:172.21.15.132:6789/0 conn(0x7f8a14161140 legacy=0x7f8a1406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.985+0000 7f8a12800640 1 -- 172.21.15.88:0/4186083300 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a140b5000 con 0x7f8a14146280 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a12800640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14146280 0x7f8a141466f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009e50 tx=0x7f8a0c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a30b2567d56b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a10a00640 1 -- 172.21.15.88:0/4186083300 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c036070 con 0x7f8a14146280 2024-04-26T20:02:56.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a10a00640 1 -- 172.21.15.88:0/4186083300 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c031600 con 0x7f8a14146280 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a10a00640 1 -- 172.21.15.88:0/4186083300 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0319d0 con 0x7f8a14146280 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a13200640 1 -- 172.21.15.88:0/4186083300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14146280 msgr2=0x7f8a141466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.986+0000 7f8a13200640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14146280 0x7f8a141466f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009e50 tx=0x7f8a0c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 -- 172.21.15.88:0/4186083300 shutdown_connections 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14146280 0x7f8a141466f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 --2- 172.21.15.88:0/4186083300 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a140b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 -- 172.21.15.88:0/4186083300 >> 172.21.15.88:0/4186083300 conn(0x7f8a140b2900 msgr2=0x7f8a140b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 -- 172.21.15.88:0/4186083300 shutdown_connections 2024-04-26T20:02:56.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.987+0000 7f8a13200640 1 -- 172.21.15.88:0/4186083300 wait complete. 2024-04-26T20:02:56.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.988+0000 7f8a13200640 1 Processor -- start 2024-04-26T20:02:56.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.988+0000 7f8a13200640 1 -- start start 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a14146280 0x7f8a14158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a1415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140cf6f0 con 0x7f8a14161140 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140cf410 con 0x7f8a14146280 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.989+0000 7f8a13200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8a140b6070 con 0x7f8a14160d60 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56088/0 (socket says 172.21.15.88:56088) 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 -- 172.21.15.88:0/3407853308 learned_addr learned my addr 172.21.15.88:0/3407853308 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 -- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 msgr2=0x7f8a1415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a1415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 -- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a14146280 msgr2=0x7f8a14158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:56.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a14146280 0x7f8a14158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 -- 172.21.15.88:0/3407853308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1415e060 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.990+0000 7f8a12800640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009e50 tx=0x7f8a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.991+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c036050 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.991+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c042070 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.991+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1413d8c0 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.991+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c009510 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.991+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a14159240 con 0x7f8a14161140 2024-04-26T20:02:56.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.992+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1413df80 con 0x7f8a14161140 2024-04-26T20:02:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.993+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f8a0c007bd0 con 0x7f8a14161140 2024-04-26T20:02:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.994+0000 7f8a0a000640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 0x7f89f406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.994+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c0acc90 con 0x7f8a14161140 2024-04-26T20:02:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.994+0000 7f8a11e00640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 0x7f89f406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.995+0000 7f8a11e00640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 0x7f89f406ee40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400a810 tx=0x7f8a04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:56.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:56.999+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a0c045360 con 0x7f8a14161140 2024-04-26T20:02:57.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.187+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8a140693c0 con 0x7f8a14161140 2024-04-26T20:02:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.188+0000 7f8a0a000640 1 -- 172.21.15.88:0/3407853308 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f8a0c07af50 con 0x7f8a14161140 2024-04-26T20:02:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 msgr2=0x7f89f406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 0x7f89f406ee40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400a810 tx=0x7f8a04008040 comp rx=0 tx=0).stop 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 msgr2=0x7f8a140cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009e50 tx=0x7f8a0c004a40 comp rx=0 tx=0).stop 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 shutdown_connections 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a14161140 0x7f8a140cfa60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f89f406c980 0x7f89f406ee40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8a14160d60 0x7f8a1415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 --2- 172.21.15.88:0/3407853308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a14146280 0x7f8a14158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 >> 172.21.15.88:0/3407853308 conn(0x7f8a140b2900 msgr2=0x7f8a1415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:57.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 shutdown_connections 2024-04-26T20:02:57.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.191+0000 7f8a13200640 1 -- 172.21.15.88:0/3407853308 wait complete. 2024-04-26T20:02:57.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol92860 --mode=777 2024-04-26T20:02:57.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.412+0000 7faf7fe00640 1 Processor -- start 2024-04-26T20:02:57.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.412+0000 7faf7fe00640 1 -- start start 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7faf80143850 0x7faf800cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7faf800cb6e0 0x7faf800cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7faf8006df60 con 0x7faf800cb6e0 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7faf80146ba0 con 0x7faf80143850 2024-04-26T20:02:57.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf8006e200 con 0x7faf80143470 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7faf800cb6e0 0x7faf800cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7faf800cb6e0 0x7faf800cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56096/0 (socket says 172.21.15.88:56096) 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7f400640 1 -- 172.21.15.88:0/2040040149 learned_addr learned my addr 172.21.15.88:0/2040040149 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.414+0000 7faf7ea00640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7e000640 1 --2- 172.21.15.88:0/2040040149 >> v2:172.21.15.132:3300/0 conn(0x7faf80143850 0x7faf800cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7f400640 1 -- 172.21.15.88:0/2040040149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 msgr2=0x7faf8006fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7f400640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8006fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7f400640 1 -- 172.21.15.88:0/2040040149 >> v2:172.21.15.132:3300/0 conn(0x7faf80143850 msgr2=0x7faf800cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7f400640 1 --2- 172.21.15.88:0/2040040149 >> v2:172.21.15.132:3300/0 conn(0x7faf80143850 0x7faf800cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7f400640 1 -- 172.21.15.88:0/2040040149 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8006e4e0 con 0x7faf800cb6e0 2024-04-26T20:02:57.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.415+0000 7faf7ea00640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8006fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7f400640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf800cb6e0 0x7faf800cbb30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf80074830 tx=0x7faf7802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36bf006590a1ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7d600640 1 -- 172.21.15.88:0/2040040149 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78036070 con 0x7faf800cb6e0 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7d600640 1 -- 172.21.15.88:0/2040040149 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf78031600 con 0x7faf800cb6e0 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7d600640 1 -- 172.21.15.88:0/2040040149 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf780319d0 con 0x7faf800cb6e0 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7fe00640 1 -- 172.21.15.88:0/2040040149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf800cb6e0 msgr2=0x7faf800cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.416+0000 7faf7fe00640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf800cb6e0 0x7faf800cbb30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faf80074830 tx=0x7faf7802a610 comp rx=0 tx=0).stop 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 -- 172.21.15.88:0/2040040149 shutdown_connections 2024-04-26T20:02:57.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf800cb6e0 0x7faf800cbb30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 --2- 172.21.15.88:0/2040040149 >> v2:172.21.15.132:3300/0 conn(0x7faf80143850 0x7faf800cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 --2- 172.21.15.88:0/2040040149 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8006fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 -- 172.21.15.88:0/2040040149 >> 172.21.15.88:0/2040040149 conn(0x7faf800d0b50 msgr2=0x7faf800d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 -- 172.21.15.88:0/2040040149 shutdown_connections 2024-04-26T20:02:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.417+0000 7faf7fe00640 1 -- 172.21.15.88:0/2040040149 wait complete. 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.418+0000 7faf7fe00640 1 Processor -- start 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.418+0000 7faf7fe00640 1 -- start start 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 0x7faf8015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8015fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf80160c20 con 0x7faf80143850 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7faf80070180 con 0x7faf800cb6e0 2024-04-26T20:02:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.419+0000 7faf7fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7faf80160da0 con 0x7faf80143470 2024-04-26T20:02:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56104/0 (socket says 172.21.15.88:56104) 2024-04-26T20:02:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 -- 172.21.15.88:0/3263950463 learned_addr learned my addr 172.21.15.88:0/3263950463 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 -- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 msgr2=0x7faf8015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7e000640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8015fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7ea00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 0x7faf8015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8015fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 -- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 msgr2=0x7faf8015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 0x7faf8015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 -- 172.21.15.88:0/3263950463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf800b59d0 con 0x7faf80143850 2024-04-26T20:02:57.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7ea00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 0x7faf8015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.420+0000 7faf7f400640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faf78009f40 tx=0x7faf78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.421+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78036050 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.421+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf78042070 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.421+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78009510 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.421+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf800b5c70 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.421+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf800b6230 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.422+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8006d1f0 con 0x7faf80143850 2024-04-26T20:02:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.423+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7faf78007bd0 con 0x7faf80143850 2024-04-26T20:02:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.424+0000 7faf76a00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 0x7faf6006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.424+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7faf780acc50 con 0x7faf80143850 2024-04-26T20:02:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.424+0000 7faf7ea00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 0x7faf6006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.425+0000 7faf7ea00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 0x7faf6006ee40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf8015d830 tx=0x7faf70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.428+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf8006d1f0 con 0x7faf80143850 2024-04-26T20:02:57.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.659+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol92860", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7faf800693c0 con 0x7faf6006c980 2024-04-26T20:02:57.674 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:57.673+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:57.674 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:57.673+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:57.674 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:57.673+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:57.674 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:57.673+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:57.674 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-04-26T20:02:57.673+0000 7f00e8400640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:02:57.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.757+0000 7faf76a00640 1 -- 172.21.15.88:0/3263950463 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf800693c0 con 0x7faf6006c980 2024-04-26T20:02:57.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.760+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 msgr2=0x7faf6006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.760+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 0x7faf6006ee40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faf8015d830 tx=0x7faf70008040 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.760+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 msgr2=0x7faf800ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.760+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faf78009f40 tx=0x7faf78004a40 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 shutdown_connections 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf80143850 0x7faf800ca730 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7faf6006c980 0x7faf6006ee40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7faf80143470 0x7faf8015fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 --2- 172.21.15.88:0/3263950463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faf800cb6e0 0x7faf8015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 >> 172.21.15.88:0/3263950463 conn(0x7faf800d0b50 msgr2=0x7faf800b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 shutdown_connections 2024-04-26T20:02:57.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.761+0000 7faf7fe00640 1 -- 172.21.15.88:0/3263950463 wait complete. 2024-04-26T20:02:57.780 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol92860 2024-04-26T20:02:57.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.978+0000 7f166a200640 1 Processor -- start 2024-04-26T20:02:57.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.979+0000 7f166a200640 1 -- start start 2024-04-26T20:02:57.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f166a200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1664156650 0x7f166415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f166a200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f166406c770 0x7f166406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f166a200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166406eea0 con 0x7f1664073120 2024-04-26T20:02:57.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f166a200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f16640b4c40 con 0x7f166406c770 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f166a200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f166406f140 con 0x7f1664156650 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f1662a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f166406c770 0x7f166406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f1662a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f166406c770 0x7f166406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56136/0 (socket says 172.21.15.88:56136) 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.980+0000 7f1662a00640 1 -- 172.21.15.88:0/943745993 learned_addr learned my addr 172.21.15.88:0/943745993 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.981+0000 7f1662a00640 1 -- 172.21.15.88:0/943745993 >> v2:172.21.15.132:3300/0 conn(0x7f1664156650 msgr2=0x7f166415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.981+0000 7f1662a00640 1 --2- 172.21.15.88:0/943745993 >> v2:172.21.15.132:3300/0 conn(0x7f1664156650 0x7f166415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.981+0000 7f1662a00640 1 -- 172.21.15.88:0/943745993 >> v1:172.21.15.132:6789/0 conn(0x7f1664073120 legacy=0x7f1664073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:02:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.981+0000 7f1662a00640 1 -- 172.21.15.88:0/943745993 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166406f420 con 0x7f166406c770 2024-04-26T20:02:57.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.982+0000 7f1662a00640 1 --2- 172.21.15.88:0/943745993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f166406cbf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1664075cd0 tx=0x7f164c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5c2c53b1569cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.982+0000 7f1662000640 1 -- 172.21.15.88:0/943745993 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c037070 con 0x7f166406c770 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.982+0000 7f1662000640 1 -- 172.21.15.88:0/943745993 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f164c032440 con 0x7f166406c770 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.982+0000 7f1662000640 1 -- 172.21.15.88:0/943745993 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c032810 con 0x7f166406c770 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 -- 172.21.15.88:0/943745993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 msgr2=0x7f166406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 --2- 172.21.15.88:0/943745993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f166406cbf0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1664075cd0 tx=0x7f164c02a230 comp rx=0 tx=0).stop 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 -- 172.21.15.88:0/943745993 shutdown_connections 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 --2- 172.21.15.88:0/943745993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f166406cbf0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 --2- 172.21.15.88:0/943745993 >> v2:172.21.15.132:3300/0 conn(0x7f1664156650 0x7f166415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 -- 172.21.15.88:0/943745993 >> 172.21.15.88:0/943745993 conn(0x7f16640b2d50 msgr2=0x7f16640b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:57.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 -- 172.21.15.88:0/943745993 shutdown_connections 2024-04-26T20:02:57.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.983+0000 7f166a200640 1 -- 172.21.15.88:0/943745993 wait complete. 2024-04-26T20:02:57.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.984+0000 7f166a200640 1 Processor -- start 2024-04-26T20:02:57.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.985+0000 7f166a200640 1 -- start start 2024-04-26T20:02:57.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1664073120 0x7f16640ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664156650 0x7f16640af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f166406fcc0 con 0x7f166406c770 2024-04-26T20:02:57.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f166406f9e0 con 0x7f1664156650 2024-04-26T20:02:57.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f166a200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f166415d320 con 0x7f1664073120 2024-04-26T20:02:57.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56138/0 (socket says 172.21.15.88:56138) 2024-04-26T20:02:57.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 -- 172.21.15.88:0/1716651258 learned_addr learned my addr 172.21.15.88:0/1716651258 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1662a00640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1664073120 0x7f16640ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 -- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1664073120 msgr2=0x7f16640ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1664073120 0x7f16640ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 -- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664156650 msgr2=0x7f16640af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664156650 0x7f16640af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.986+0000 7f1663400640 1 -- 172.21.15.88:0/1716651258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1664142ce0 con 0x7f166406c770 2024-04-26T20:02:57.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f1663400640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f165400b410 tx=0x7f165400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654015020 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1654011850 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654011c40 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166415a520 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.987+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166415aae0 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.988+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1664069650 con 0x7f166406c770 2024-04-26T20:02:57.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.989+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1654018020 con 0x7f166406c770 2024-04-26T20:02:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.990+0000 7f165b400640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 0x7f164006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:02:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.990+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f165408bb10 con 0x7f166406c770 2024-04-26T20:02:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.990+0000 7f1662a00640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 0x7f164006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:02:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.991+0000 7f1662a00640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 0x7f164006ee40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f164c034000 tx=0x7f164c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:02:57.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:57.995+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f165401f020 con 0x7f166406c770 2024-04-26T20:02:58.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.225+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol92860", "target": ["mon-mgr", ""]}) -- 0x7f166415cbc0 con 0x7f164006c980 2024-04-26T20:02:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.236+0000 7f165b400640 1 -- 172.21.15.88:0/1716651258 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f166415cbc0 con 0x7f164006c980 2024-04-26T20:02:58.236 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 msgr2=0x7f164006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 0x7f164006ee40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f164c034000 tx=0x7f164c02e000 comp rx=0 tx=0).stop 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 msgr2=0x7f16641427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f165400b410 tx=0x7f165400b8d0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 shutdown_connections 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1664156650 0x7f16640af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f164006c980 0x7f164006ee40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1664073120 0x7f16640ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 --2- 172.21.15.88:0/1716651258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406c770 0x7f16641427a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:02:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.238+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 >> 172.21.15.88:0/1716651258 conn(0x7f16640b2d50 msgr2=0x7f16640cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:02:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.239+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 shutdown_connections 2024-04-26T20:02:58.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:02:58.239+0000 7f166a200640 1 -- 172.21.15.88:0/1716651258 wait complete. 2024-04-26T20:02:58.257 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol92860 with 3000 files each 0MB size under directory volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece 2024-04-26T20:02:58.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.258 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:58.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:58.307 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:58.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:58.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:58.446 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000408728 s, 0.0 kB/s 2024-04-26T20:02:58.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:58.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:58.504 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:58.565 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183828 s, 0.0 kB/s 2024-04-26T20:02:58.566 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.566 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:58.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:58.615 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:58.729 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183228 s, 0.0 kB/s 2024-04-26T20:02:58.731 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.731 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:58.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:58.785 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:58.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:58.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:58.847 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312454 s, 0.0 kB/s 2024-04-26T20:02:58.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:58.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:58.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:58.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:58.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:58.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184508 s, 0.0 kB/s 2024-04-26T20:02:58.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:58.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.087 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237512 s, 0.0 kB/s 2024-04-26T20:02:59.088 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.141 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.207 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187394 s, 0.0 kB/s 2024-04-26T20:02:59.208 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.208 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.263 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.335 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282428 s, 0.0 kB/s 2024-04-26T20:02:59.336 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.336 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.388 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020948 s, 0.0 kB/s 2024-04-26T20:02:59.498 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.498 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.553 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.620 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181482 s, 0.0 kB/s 2024-04-26T20:02:59.622 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.622 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203243 s, 0.0 kB/s 2024-04-26T20:02:59.745 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.745 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167494 s, 0.0 kB/s 2024-04-26T20:02:59.870 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.871 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:02:59.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:02:59.925 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:02:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:02:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:02:59.991 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220646 s, 0.0 kB/s 2024-04-26T20:02:59.992 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:02:59.992 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.045 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.113 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018228 s, 0.0 kB/s 2024-04-26T20:03:00.114 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.114 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.169 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.237 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192566 s, 0.0 kB/s 2024-04-26T20:03:00.238 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.238 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.293 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.355 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000136104 s, 0.0 kB/s 2024-04-26T20:03:00.357 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.357 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.408 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.471 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195978 s, 0.0 kB/s 2024-04-26T20:03:00.472 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.472 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.526 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.591 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161552 s, 0.0 kB/s 2024-04-26T20:03:00.592 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.592 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.749 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.750 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243832 s, 0.0 kB/s 2024-04-26T20:03:00.751 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.751 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.808 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:00.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:00.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:00.880 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306435 s, 0.0 kB/s 2024-04-26T20:03:00.882 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:00.882 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:00.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:00.939 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.010 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002041 s, 0.0 kB/s 2024-04-26T20:03:01.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.239 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293242 s, 0.0 kB/s 2024-04-26T20:03:01.310 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.310 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.365 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.426 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177107 s, 0.0 kB/s 2024-04-26T20:03:01.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.427 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.478 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.590 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176097 s, 0.0 kB/s 2024-04-26T20:03:01.591 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.591 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.755 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231603 s, 0.0 kB/s 2024-04-26T20:03:01.757 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.757 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.812 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.876 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205018 s, 0.0 kB/s 2024-04-26T20:03:01.877 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:01.878 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:01.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:01.931 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:01.998 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:01.998 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:01.998 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172054 s, 0.0 kB/s 2024-04-26T20:03:01.999 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.000 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.053 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.117 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000317693 s, 0.0 kB/s 2024-04-26T20:03:02.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.119 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190952 s, 0.0 kB/s 2024-04-26T20:03:02.240 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.240 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.293 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.358 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00033601 s, 0.0 kB/s 2024-04-26T20:03:02.359 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.479 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256694 s, 0.0 kB/s 2024-04-26T20:03:02.481 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.481 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.529 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233994 s, 0.0 kB/s 2024-04-26T20:03:02.639 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.639 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.693 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.763 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279412 s, 0.0 kB/s 2024-04-26T20:03:02.764 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.764 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.821 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:02.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215448 s, 0.0 kB/s 2024-04-26T20:03:02.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:02.890 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:02.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:02.945 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200572 s, 0.0 kB/s 2024-04-26T20:03:03.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157837 s, 0.0 kB/s 2024-04-26T20:03:03.134 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.134 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.186 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.300 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197884 s, 0.0 kB/s 2024-04-26T20:03:03.301 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.301 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.354 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.418 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182004 s, 0.0 kB/s 2024-04-26T20:03:03.419 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.419 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.535 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154997 s, 0.0 kB/s 2024-04-26T20:03:03.536 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.536 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.657 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020682 s, 0.0 kB/s 2024-04-26T20:03:03.658 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.658 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.713 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.775 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.776 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165107 s, 0.0 kB/s 2024-04-26T20:03:03.777 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.777 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.825 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:03.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:03.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:03.937 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217113 s, 0.0 kB/s 2024-04-26T20:03:03.938 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:03.938 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:03.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:03.991 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018972 s, 0.0 kB/s 2024-04-26T20:03:04.059 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.059 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.113 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.177 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275243 s, 0.0 kB/s 2024-04-26T20:03:04.178 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.231 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.291 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201034 s, 0.0 kB/s 2024-04-26T20:03:04.292 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.292 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.345 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.413 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.413 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.413 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287427 s, 0.0 kB/s 2024-04-26T20:03:04.414 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.414 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.539 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.539 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220632 s, 0.0 kB/s 2024-04-26T20:03:04.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.540 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.595 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219056 s, 0.0 kB/s 2024-04-26T20:03:04.666 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.666 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.717 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.830 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293044 s, 0.0 kB/s 2024-04-26T20:03:04.831 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.831 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:04.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:04.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:04.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284305 s, 0.0 kB/s 2024-04-26T20:03:04.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:04.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.004 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.071 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158408 s, 0.0 kB/s 2024-04-26T20:03:05.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.072 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000737456 s, 0.0 kB/s 2024-04-26T20:03:05.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.243 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.308 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223473 s, 0.0 kB/s 2024-04-26T20:03:05.309 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.309 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.362 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.426 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.426 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.426 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269542 s, 0.0 kB/s 2024-04-26T20:03:05.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.427 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.592 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.592 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.592 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217999 s, 0.0 kB/s 2024-04-26T20:03:05.593 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.593 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.649 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000334395 s, 0.0 kB/s 2024-04-26T20:03:05.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.719 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.835 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018542 s, 0.0 kB/s 2024-04-26T20:03:05.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.836 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148358 s, 0.0 kB/s 2024-04-26T20:03:05.947 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:05.948 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:05.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:05.988 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203865 s, 0.0 kB/s 2024-04-26T20:03:06.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.122 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.213 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.213 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.213 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152875 s, 0.0 kB/s 2024-04-26T20:03:06.214 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.215 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.253 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215531 s, 0.0 kB/s 2024-04-26T20:03:06.358 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.358 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.410 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.475 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202782 s, 0.0 kB/s 2024-04-26T20:03:06.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.598 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000311678 s, 0.0 kB/s 2024-04-26T20:03:06.599 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.599 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.656 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.725 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258468 s, 0.0 kB/s 2024-04-26T20:03:06.727 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.727 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.782 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189208 s, 0.0 kB/s 2024-04-26T20:03:06.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:06.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:06.904 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:06.972 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:06.972 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:06.972 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160072 s, 0.0 kB/s 2024-04-26T20:03:06.973 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:06.973 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000310669 s, 0.0 kB/s 2024-04-26T20:03:07.131 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.131 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.253 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.253 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000326634 s, 0.0 kB/s 2024-04-26T20:03:07.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.254 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.305 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.371 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169925 s, 0.0 kB/s 2024-04-26T20:03:07.372 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.372 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.427 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.493 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.493 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.493 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025519 s, 0.0 kB/s 2024-04-26T20:03:07.494 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.494 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.541 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00033795 s, 0.0 kB/s 2024-04-26T20:03:07.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.657 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.713 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.778 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312601 s, 0.0 kB/s 2024-04-26T20:03:07.779 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.779 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.825 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:07.931 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:07.931 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:07.931 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227856 s, 0.0 kB/s 2024-04-26T20:03:07.932 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:07.932 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:07.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:07.989 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.054 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029331 s, 0.0 kB/s 2024-04-26T20:03:08.056 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.056 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.111 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.176 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251069 s, 0.0 kB/s 2024-04-26T20:03:08.177 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.177 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020713 s, 0.0 kB/s 2024-04-26T20:03:08.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.288 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.340 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207214 s, 0.0 kB/s 2024-04-26T20:03:08.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.460 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.526 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206672 s, 0.0 kB/s 2024-04-26T20:03:08.527 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.528 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.582 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.646 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.646 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.646 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240577 s, 0.0 kB/s 2024-04-26T20:03:08.647 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.647 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.700 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235113 s, 0.0 kB/s 2024-04-26T20:03:08.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.814 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237427 s, 0.0 kB/s 2024-04-26T20:03:08.880 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:08.880 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:08.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:08.934 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:08.998 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225395 s, 0.0 kB/s 2024-04-26T20:03:09.000 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.000 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.054 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.118 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.118 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.118 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197595 s, 0.0 kB/s 2024-04-26T20:03:09.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.119 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.240 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201415 s, 0.0 kB/s 2024-04-26T20:03:09.241 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.241 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.292 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242331 s, 0.0 kB/s 2024-04-26T20:03:09.357 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.358 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.409 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.469 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.469 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192086 s, 0.0 kB/s 2024-04-26T20:03:09.470 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.470 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.522 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.634 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253167 s, 0.0 kB/s 2024-04-26T20:03:09.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.693 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240832 s, 0.0 kB/s 2024-04-26T20:03:09.761 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.761 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.815 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164808 s, 0.0 kB/s 2024-04-26T20:03:09.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:09.887 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:09.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:09.944 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187332 s, 0.0 kB/s 2024-04-26T20:03:10.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.014 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.067 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.135 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194472 s, 0.0 kB/s 2024-04-26T20:03:10.136 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.136 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.179 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:03:10.189 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:03:10.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.191 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184913 s, 0.0 kB/s 2024-04-26T20:03:10.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.443 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.444 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025578 s, 0.0 kB/s 2024-04-26T20:03:10.445 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.445 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.502 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.564 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233305 s, 0.0 kB/s 2024-04-26T20:03:10.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.683 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180385 s, 0.0 kB/s 2024-04-26T20:03:10.684 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.684 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.803 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.804 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245296 s, 0.0 kB/s 2024-04-26T20:03:10.805 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.859 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:10.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:10.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:10.928 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212668 s, 0.0 kB/s 2024-04-26T20:03:10.929 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:10.930 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:10.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:10.987 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.051 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273928 s, 0.0 kB/s 2024-04-26T20:03:11.052 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.052 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.104 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.167 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.167 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.167 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241226 s, 0.0 kB/s 2024-04-26T20:03:11.168 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.168 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.223 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.290 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206941 s, 0.0 kB/s 2024-04-26T20:03:11.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.291 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.344 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.410 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025747 s, 0.0 kB/s 2024-04-26T20:03:11.411 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.412 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.468 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.534 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228447 s, 0.0 kB/s 2024-04-26T20:03:11.535 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.535 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.589 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.654 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000401384 s, 0.0 kB/s 2024-04-26T20:03:11.656 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.656 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.707 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.771 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.771 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238987 s, 0.0 kB/s 2024-04-26T20:03:11.773 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.773 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.826 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:11.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:11.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:11.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024319 s, 0.0 kB/s 2024-04-26T20:03:11.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:11.890 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:11.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:11.940 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263842 s, 0.0 kB/s 2024-04-26T20:03:12.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.054 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.176 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.177 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291636 s, 0.0 kB/s 2024-04-26T20:03:12.178 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.230 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.295 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000285321 s, 0.0 kB/s 2024-04-26T20:03:12.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.296 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.347 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216138 s, 0.0 kB/s 2024-04-26T20:03:12.414 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.414 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.465 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.529 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.529 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.529 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269059 s, 0.0 kB/s 2024-04-26T20:03:12.531 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.531 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.585 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214275 s, 0.0 kB/s 2024-04-26T20:03:12.650 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.650 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.699 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179192 s, 0.0 kB/s 2024-04-26T20:03:12.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.803 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:12.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:12.850 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:12.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:12.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:12.960 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227605 s, 0.0 kB/s 2024-04-26T20:03:12.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:12.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226589 s, 0.0 kB/s 2024-04-26T20:03:13.083 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.083 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.138 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.202 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270266 s, 0.0 kB/s 2024-04-26T20:03:13.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.203 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.252 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.364 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218822 s, 0.0 kB/s 2024-04-26T20:03:13.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.365 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.419 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.483 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221078 s, 0.0 kB/s 2024-04-26T20:03:13.485 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.485 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.534 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.642 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.642 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.642 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246673 s, 0.0 kB/s 2024-04-26T20:03:13.643 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.643 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.700 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.763 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.763 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018629 s, 0.0 kB/s 2024-04-26T20:03:13.764 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.764 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.809 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:13.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:13.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:13.923 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255211 s, 0.0 kB/s 2024-04-26T20:03:13.925 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:13.925 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:13.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:13.979 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.048 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236879 s, 0.0 kB/s 2024-04-26T20:03:14.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.050 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.102 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254001 s, 0.0 kB/s 2024-04-26T20:03:14.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.170 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.223 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.292 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.292 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.292 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221943 s, 0.0 kB/s 2024-04-26T20:03:14.293 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.293 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.347 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260023 s, 0.0 kB/s 2024-04-26T20:03:14.416 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.416 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185097 s, 0.0 kB/s 2024-04-26T20:03:14.537 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.537 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.592 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.664 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173384 s, 0.0 kB/s 2024-04-26T20:03:14.665 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.665 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.718 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207306 s, 0.0 kB/s 2024-04-26T20:03:14.782 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.782 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:14.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:14.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:14.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:14.949 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000297081 s, 0.0 kB/s 2024-04-26T20:03:14.950 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:14.950 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.006 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.076 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227475 s, 0.0 kB/s 2024-04-26T20:03:15.077 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.077 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.132 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.199 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000332762 s, 0.0 kB/s 2024-04-26T20:03:15.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.200 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.318 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.318 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.319 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247462 s, 0.0 kB/s 2024-04-26T20:03:15.320 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.320 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.374 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024262 s, 0.0 kB/s 2024-04-26T20:03:15.447 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.504 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280216 s, 0.0 kB/s 2024-04-26T20:03:15.568 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.568 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.625 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.691 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247276 s, 0.0 kB/s 2024-04-26T20:03:15.692 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.692 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.746 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.812 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287081 s, 0.0 kB/s 2024-04-26T20:03:15.814 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.814 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.868 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:15.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:15.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:15.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175828 s, 0.0 kB/s 2024-04-26T20:03:15.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:15.928 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:15.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:15.977 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234061 s, 0.0 kB/s 2024-04-26T20:03:16.039 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.039 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.093 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.157 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240709 s, 0.0 kB/s 2024-04-26T20:03:16.159 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.159 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024958 s, 0.0 kB/s 2024-04-26T20:03:16.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.277 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.329 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251853 s, 0.0 kB/s 2024-04-26T20:03:16.400 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.400 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000316218 s, 0.0 kB/s 2024-04-26T20:03:16.528 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.528 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.585 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265829 s, 0.0 kB/s 2024-04-26T20:03:16.651 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.707 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.774 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230992 s, 0.0 kB/s 2024-04-26T20:03:16.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.831 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000341013 s, 0.0 kB/s 2024-04-26T20:03:16.902 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:16.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:16.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:16.956 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.015 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171768 s, 0.0 kB/s 2024-04-26T20:03:17.016 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.016 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.073 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.139 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196878 s, 0.0 kB/s 2024-04-26T20:03:17.140 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.140 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.196 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.262 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156066 s, 0.0 kB/s 2024-04-26T20:03:17.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.313 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.376 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.376 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.376 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184238 s, 0.0 kB/s 2024-04-26T20:03:17.377 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.377 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.433 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.495 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.495 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.496 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214473 s, 0.0 kB/s 2024-04-26T20:03:17.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.497 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.551 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.619 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.619 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159303 s, 0.0 kB/s 2024-04-26T20:03:17.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.620 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.672 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.732 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226136 s, 0.0 kB/s 2024-04-26T20:03:17.733 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.733 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.782 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:17.893 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187765 s, 0.0 kB/s 2024-04-26T20:03:17.894 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:17.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:17.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:17.948 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.017 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014509 s, 0.0 kB/s 2024-04-26T20:03:18.018 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.073 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.139 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239723 s, 0.0 kB/s 2024-04-26T20:03:18.140 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.140 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.197 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.262 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196354 s, 0.0 kB/s 2024-04-26T20:03:18.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.316 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150116 s, 0.0 kB/s 2024-04-26T20:03:18.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.381 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.433 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022108 s, 0.0 kB/s 2024-04-26T20:03:18.549 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.550 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.666 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217497 s, 0.0 kB/s 2024-04-26T20:03:18.667 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.667 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.722 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.786 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193346 s, 0.0 kB/s 2024-04-26T20:03:18.787 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.787 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.841 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:18.908 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:18.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:18.909 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261267 s, 0.0 kB/s 2024-04-26T20:03:18.910 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:18.911 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:18.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:18.967 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.033 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254433 s, 0.0 kB/s 2024-04-26T20:03:19.035 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.035 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.087 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.149 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196124 s, 0.0 kB/s 2024-04-26T20:03:19.150 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.150 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.202 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020994 s, 0.0 kB/s 2024-04-26T20:03:19.314 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.314 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.434 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254988 s, 0.0 kB/s 2024-04-26T20:03:19.436 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.436 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.486 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229853 s, 0.0 kB/s 2024-04-26T20:03:19.549 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.549 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.597 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.702 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231105 s, 0.0 kB/s 2024-04-26T20:03:19.703 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.703 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.760 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.831 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229392 s, 0.0 kB/s 2024-04-26T20:03:19.832 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.833 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:19.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:19.887 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287005 s, 0.0 kB/s 2024-04-26T20:03:19.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:19.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.005 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.116 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.116 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.116 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204501 s, 0.0 kB/s 2024-04-26T20:03:20.117 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.117 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.237 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193664 s, 0.0 kB/s 2024-04-26T20:03:20.238 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.238 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.290 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209022 s, 0.0 kB/s 2024-04-26T20:03:20.357 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.357 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.483 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193454 s, 0.0 kB/s 2024-04-26T20:03:20.484 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.540 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.606 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176858 s, 0.0 kB/s 2024-04-26T20:03:20.607 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.607 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.660 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.723 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238966 s, 0.0 kB/s 2024-04-26T20:03:20.724 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.779 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:20.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:20.842 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:20.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221022 s, 0.0 kB/s 2024-04-26T20:03:20.844 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:20.844 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:20.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:20.894 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000296205 s, 0.0 kB/s 2024-04-26T20:03:21.008 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.008 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.065 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215752 s, 0.0 kB/s 2024-04-26T20:03:21.131 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.131 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.183 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023972 s, 0.0 kB/s 2024-04-26T20:03:21.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.296 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.350 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.420 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163991 s, 0.0 kB/s 2024-04-26T20:03:21.421 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.421 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.473 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.584 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231176 s, 0.0 kB/s 2024-04-26T20:03:21.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.709 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000323394 s, 0.0 kB/s 2024-04-26T20:03:21.711 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.763 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252775 s, 0.0 kB/s 2024-04-26T20:03:21.876 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:21.877 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:21.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:21.933 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:21.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:21.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:21.999 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157133 s, 0.0 kB/s 2024-04-26T20:03:22.000 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.000 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.053 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.119 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163531 s, 0.0 kB/s 2024-04-26T20:03:22.120 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.174 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.286 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002547 s, 0.0 kB/s 2024-04-26T20:03:22.287 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.287 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.409 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219855 s, 0.0 kB/s 2024-04-26T20:03:22.410 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.410 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.463 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.526 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294385 s, 0.0 kB/s 2024-04-26T20:03:22.527 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.528 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.584 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191126 s, 0.0 kB/s 2024-04-26T20:03:22.652 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.652 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.706 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.774 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280143 s, 0.0 kB/s 2024-04-26T20:03:22.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.828 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:22.892 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:22.892 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:22.892 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225113 s, 0.0 kB/s 2024-04-26T20:03:22.893 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:22.893 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:22.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:22.945 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181508 s, 0.0 kB/s 2024-04-26T20:03:23.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.054 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.171 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.171 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.171 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180914 s, 0.0 kB/s 2024-04-26T20:03:23.172 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.173 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.227 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.289 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.290 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021367 s, 0.0 kB/s 2024-04-26T20:03:23.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.291 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.344 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194122 s, 0.0 kB/s 2024-04-26T20:03:23.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.412 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.462 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.572 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.572 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204719 s, 0.0 kB/s 2024-04-26T20:03:23.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306083 s, 0.0 kB/s 2024-04-26T20:03:23.695 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.695 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.809 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.809 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193249 s, 0.0 kB/s 2024-04-26T20:03:23.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.811 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.863 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:23.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:23.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:23.924 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000387365 s, 0.0 kB/s 2024-04-26T20:03:23.926 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:23.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:23.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:23.981 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.047 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257364 s, 0.0 kB/s 2024-04-26T20:03:24.048 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.048 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.100 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.167 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.167 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.167 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284032 s, 0.0 kB/s 2024-04-26T20:03:24.168 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.169 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.227 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.288 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286542 s, 0.0 kB/s 2024-04-26T20:03:24.289 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.340 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291939 s, 0.0 kB/s 2024-04-26T20:03:24.454 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.454 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.505 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.570 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198068 s, 0.0 kB/s 2024-04-26T20:03:24.571 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.571 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.623 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.690 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185212 s, 0.0 kB/s 2024-04-26T20:03:24.692 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.692 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.747 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187465 s, 0.0 kB/s 2024-04-26T20:03:24.817 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.817 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:24.933 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:24.933 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:24.933 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018779 s, 0.0 kB/s 2024-04-26T20:03:24.934 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:24.934 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:24.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:24.986 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.051 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030344 s, 0.0 kB/s 2024-04-26T20:03:25.052 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.052 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.105 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257183 s, 0.0 kB/s 2024-04-26T20:03:25.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.170 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.225 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.291 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.291 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.291 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272197 s, 0.0 kB/s 2024-04-26T20:03:25.292 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.292 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.347 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.415 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.415 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216161 s, 0.0 kB/s 2024-04-26T20:03:25.417 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.417 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.473 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.542 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.543 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.543 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222285 s, 0.0 kB/s 2024-04-26T20:03:25.544 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.544 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.596 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.657 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249046 s, 0.0 kB/s 2024-04-26T20:03:25.658 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.658 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.714 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.776 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266945 s, 0.0 kB/s 2024-04-26T20:03:25.777 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.777 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.830 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:25.940 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231345 s, 0.0 kB/s 2024-04-26T20:03:25.941 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:25.942 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:25.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:25.994 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.057 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.057 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.057 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263021 s, 0.0 kB/s 2024-04-26T20:03:26.058 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.058 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.221 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271742 s, 0.0 kB/s 2024-04-26T20:03:26.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.222 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.275 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025613 s, 0.0 kB/s 2024-04-26T20:03:26.341 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.341 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.394 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.463 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000321651 s, 0.0 kB/s 2024-04-26T20:03:26.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.464 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.518 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212386 s, 0.0 kB/s 2024-04-26T20:03:26.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.701 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212082 s, 0.0 kB/s 2024-04-26T20:03:26.702 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.702 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.825 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000325298 s, 0.0 kB/s 2024-04-26T20:03:26.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.826 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:26.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:26.881 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:26.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:26.948 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030395 s, 0.0 kB/s 2024-04-26T20:03:26.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:26.949 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.006 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.075 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000341878 s, 0.0 kB/s 2024-04-26T20:03:27.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.131 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162722 s, 0.0 kB/s 2024-04-26T20:03:27.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.198 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.314 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173579 s, 0.0 kB/s 2024-04-26T20:03:27.316 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.316 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.365 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.433 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.433 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.433 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312214 s, 0.0 kB/s 2024-04-26T20:03:27.434 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.435 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.491 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.562 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240199 s, 0.0 kB/s 2024-04-26T20:03:27.563 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.563 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.620 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.683 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154189 s, 0.0 kB/s 2024-04-26T20:03:27.684 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.684 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.803 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.804 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000347925 s, 0.0 kB/s 2024-04-26T20:03:27.805 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.858 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:27.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:27.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:27.923 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026819 s, 0.0 kB/s 2024-04-26T20:03:27.925 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:27.925 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:27.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:27.980 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.045 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161614 s, 0.0 kB/s 2024-04-26T20:03:28.046 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.046 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.163 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000296111 s, 0.0 kB/s 2024-04-26T20:03:28.164 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.165 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.215 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.326 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234555 s, 0.0 kB/s 2024-04-26T20:03:28.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.381 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272467 s, 0.0 kB/s 2024-04-26T20:03:28.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.501 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248185 s, 0.0 kB/s 2024-04-26T20:03:28.614 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.614 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.667 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025473 s, 0.0 kB/s 2024-04-26T20:03:28.735 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.735 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222448 s, 0.0 kB/s 2024-04-26T20:03:28.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:28.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:28.904 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:28.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:28.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:28.974 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022748 s, 0.0 kB/s 2024-04-26T20:03:28.975 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:28.975 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.032 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.098 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173008 s, 0.0 kB/s 2024-04-26T20:03:29.099 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.099 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.152 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.212 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000130906 s, 0.0 kB/s 2024-04-26T20:03:29.213 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.213 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.261 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000320105 s, 0.0 kB/s 2024-04-26T20:03:29.373 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.373 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.430 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.497 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195682 s, 0.0 kB/s 2024-04-26T20:03:29.498 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.499 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.549 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.612 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.612 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.612 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151218 s, 0.0 kB/s 2024-04-26T20:03:29.613 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.613 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.666 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199831 s, 0.0 kB/s 2024-04-26T20:03:29.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.788 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.859 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273206 s, 0.0 kB/s 2024-04-26T20:03:29.860 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:29.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:29.916 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:29.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:29.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:29.980 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190332 s, 0.0 kB/s 2024-04-26T20:03:29.981 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:29.981 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.033 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.098 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000311001 s, 0.0 kB/s 2024-04-26T20:03:30.098 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.099 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.155 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.216 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153081 s, 0.0 kB/s 2024-04-26T20:03:30.217 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.217 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.264 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030258 s, 0.0 kB/s 2024-04-26T20:03:30.380 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.435 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.505 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185951 s, 0.0 kB/s 2024-04-26T20:03:30.506 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.506 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.556 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.667 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.667 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.667 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231256 s, 0.0 kB/s 2024-04-26T20:03:30.669 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.788 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242049 s, 0.0 kB/s 2024-04-26T20:03:30.788 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.841 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:30.908 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:30.909 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231066 s, 0.0 kB/s 2024-04-26T20:03:30.910 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:30.910 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:30.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:30.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.031 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.031 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202956 s, 0.0 kB/s 2024-04-26T20:03:31.032 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.032 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.084 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.153 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182815 s, 0.0 kB/s 2024-04-26T20:03:31.154 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.154 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.208 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.267 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218619 s, 0.0 kB/s 2024-04-26T20:03:31.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.382 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278968 s, 0.0 kB/s 2024-04-26T20:03:31.383 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.438 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.503 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.503 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.504 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241714 s, 0.0 kB/s 2024-04-26T20:03:31.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.505 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.559 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180779 s, 0.0 kB/s 2024-04-26T20:03:31.624 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.624 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.678 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267807 s, 0.0 kB/s 2024-04-26T20:03:31.742 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.743 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.808 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021964 s, 0.0 kB/s 2024-04-26T20:03:31.906 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:31.906 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:31.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:31.959 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.026 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.027 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226489 s, 0.0 kB/s 2024-04-26T20:03:32.028 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.083 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.143 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199272 s, 0.0 kB/s 2024-04-26T20:03:32.144 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.144 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.194 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.302 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223621 s, 0.0 kB/s 2024-04-26T20:03:32.303 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.303 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.357 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.421 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.421 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.421 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218021 s, 0.0 kB/s 2024-04-26T20:03:32.422 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.422 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.471 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.537 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000324752 s, 0.0 kB/s 2024-04-26T20:03:32.538 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.538 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.658 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204486 s, 0.0 kB/s 2024-04-26T20:03:32.659 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.660 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.711 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.818 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171425 s, 0.0 kB/s 2024-04-26T20:03:32.819 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.819 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:32.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:32.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:32.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:32.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:32.978 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029769 s, 0.0 kB/s 2024-04-26T20:03:32.979 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:32.979 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.032 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.095 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026376 s, 0.0 kB/s 2024-04-26T20:03:33.096 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.096 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.146 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000321915 s, 0.0 kB/s 2024-04-26T20:03:33.260 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.260 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.315 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256041 s, 0.0 kB/s 2024-04-26T20:03:33.383 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.436 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.507 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231067 s, 0.0 kB/s 2024-04-26T20:03:33.508 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.508 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.561 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257508 s, 0.0 kB/s 2024-04-26T20:03:33.626 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.626 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015168 s, 0.0 kB/s 2024-04-26T20:03:33.745 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.745 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.795 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:33.899 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:33.899 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:33.899 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194562 s, 0.0 kB/s 2024-04-26T20:03:33.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:33.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:33.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:33.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.018 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.018 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173059 s, 0.0 kB/s 2024-04-26T20:03:34.019 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.019 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.141 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.142 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242704 s, 0.0 kB/s 2024-04-26T20:03:34.143 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.143 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.198 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.263 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018113 s, 0.0 kB/s 2024-04-26T20:03:34.264 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.264 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.318 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.386 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021529 s, 0.0 kB/s 2024-04-26T20:03:34.387 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.387 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.550 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000299883 s, 0.0 kB/s 2024-04-26T20:03:34.551 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.551 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.608 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.677 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000332284 s, 0.0 kB/s 2024-04-26T20:03:34.678 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.733 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.799 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151196 s, 0.0 kB/s 2024-04-26T20:03:34.800 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.800 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182398 s, 0.0 kB/s 2024-04-26T20:03:34.919 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:34.920 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:34.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:34.971 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.086 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191421 s, 0.0 kB/s 2024-04-26T20:03:35.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.087 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.142 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.254 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.255 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.255 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258284 s, 0.0 kB/s 2024-04-26T20:03:35.256 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.256 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.312 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204952 s, 0.0 kB/s 2024-04-26T20:03:35.377 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.377 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.430 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.496 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214156 s, 0.0 kB/s 2024-04-26T20:03:35.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.497 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.552 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.616 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197551 s, 0.0 kB/s 2024-04-26T20:03:35.618 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.618 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.671 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.735 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.736 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184499 s, 0.0 kB/s 2024-04-26T20:03:35.737 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.737 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:35.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:35.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:35.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189189 s, 0.0 kB/s 2024-04-26T20:03:35.902 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:35.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:35.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:35.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182078 s, 0.0 kB/s 2024-04-26T20:03:36.020 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.020 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.138 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211594 s, 0.0 kB/s 2024-04-26T20:03:36.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.191 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.257 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244587 s, 0.0 kB/s 2024-04-26T20:03:36.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.258 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.312 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.381 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.381 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.381 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202733 s, 0.0 kB/s 2024-04-26T20:03:36.382 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.382 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.432 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.544 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231371 s, 0.0 kB/s 2024-04-26T20:03:36.545 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.546 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.666 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226346 s, 0.0 kB/s 2024-04-26T20:03:36.670 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.670 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.726 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205595 s, 0.0 kB/s 2024-04-26T20:03:36.794 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.794 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.848 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:36.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:36.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:36.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192488 s, 0.0 kB/s 2024-04-26T20:03:36.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:36.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:36.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:36.969 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.037 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239447 s, 0.0 kB/s 2024-04-26T20:03:37.039 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.039 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.093 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.157 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160675 s, 0.0 kB/s 2024-04-26T20:03:37.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.159 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.210 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.273 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169534 s, 0.0 kB/s 2024-04-26T20:03:37.274 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.274 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.326 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.393 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.393 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.393 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156933 s, 0.0 kB/s 2024-04-26T20:03:37.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.394 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.445 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000391249 s, 0.0 kB/s 2024-04-26T20:03:37.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.570 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.637 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262755 s, 0.0 kB/s 2024-04-26T20:03:37.639 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.639 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.694 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.764 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183386 s, 0.0 kB/s 2024-04-26T20:03:37.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.766 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.816 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002995 s, 0.0 kB/s 2024-04-26T20:03:37.925 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:37.925 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:37.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:37.981 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.042 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157986 s, 0.0 kB/s 2024-04-26T20:03:38.043 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.043 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.093 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.203 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.204 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192412 s, 0.0 kB/s 2024-04-26T20:03:38.205 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.259 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187052 s, 0.0 kB/s 2024-04-26T20:03:38.324 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.324 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.380 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.448 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230942 s, 0.0 kB/s 2024-04-26T20:03:38.449 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.500 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.567 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268449 s, 0.0 kB/s 2024-04-26T20:03:38.568 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.568 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.721 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.721 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.721 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175144 s, 0.0 kB/s 2024-04-26T20:03:38.722 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.722 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:38.880 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:38.880 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:38.881 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217072 s, 0.0 kB/s 2024-04-26T20:03:38.882 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:38.882 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:38.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:38.938 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.005 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238221 s, 0.0 kB/s 2024-04-26T20:03:39.006 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.006 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.060 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000332132 s, 0.0 kB/s 2024-04-26T20:03:39.130 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.130 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237894 s, 0.0 kB/s 2024-04-26T20:03:39.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.254 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.307 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.369 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.369 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.369 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191331 s, 0.0 kB/s 2024-04-26T20:03:39.370 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.370 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.420 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.483 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016862 s, 0.0 kB/s 2024-04-26T20:03:39.484 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.536 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.599 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.599 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245164 s, 0.0 kB/s 2024-04-26T20:03:39.600 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.600 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.719 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002509 s, 0.0 kB/s 2024-04-26T20:03:39.720 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.770 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233197 s, 0.0 kB/s 2024-04-26T20:03:39.882 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:39.882 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:39.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:39.936 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.008 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251854 s, 0.0 kB/s 2024-04-26T20:03:40.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.009 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232453 s, 0.0 kB/s 2024-04-26T20:03:40.132 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.132 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.184 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182424 s, 0.0 kB/s 2024-04-26T20:03:40.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.254 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.305 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.318 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:03:40.357 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:03:40.408 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.408 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.408 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208359 s, 0.0 kB/s 2024-04-26T20:03:40.409 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.409 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.545 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200666 s, 0.0 kB/s 2024-04-26T20:03:40.616 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.616 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.665 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.777 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000348396 s, 0.0 kB/s 2024-04-26T20:03:40.778 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.778 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.830 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:40.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:40.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:40.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203759 s, 0.0 kB/s 2024-04-26T20:03:40.902 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:40.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:40.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:40.958 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201544 s, 0.0 kB/s 2024-04-26T20:03:41.026 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.026 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.139 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151461 s, 0.0 kB/s 2024-04-26T20:03:41.141 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.141 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.193 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.259 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.259 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196379 s, 0.0 kB/s 2024-04-26T20:03:41.260 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.260 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.316 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.380 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000300098 s, 0.0 kB/s 2024-04-26T20:03:41.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.381 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.505 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000317446 s, 0.0 kB/s 2024-04-26T20:03:41.506 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.506 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.559 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.622 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196794 s, 0.0 kB/s 2024-04-26T20:03:41.623 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.623 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228195 s, 0.0 kB/s 2024-04-26T20:03:41.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.744 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.796 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.856 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187035 s, 0.0 kB/s 2024-04-26T20:03:41.858 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.858 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:41.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:41.908 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:41.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:41.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:41.973 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212716 s, 0.0 kB/s 2024-04-26T20:03:41.974 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:41.975 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.026 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.087 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274084 s, 0.0 kB/s 2024-04-26T20:03:42.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.143 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.208 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264707 s, 0.0 kB/s 2024-04-26T20:03:42.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.210 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.265 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206267 s, 0.0 kB/s 2024-04-26T20:03:42.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.387 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.457 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255759 s, 0.0 kB/s 2024-04-26T20:03:42.459 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.459 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.513 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.577 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224495 s, 0.0 kB/s 2024-04-26T20:03:42.578 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.579 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.634 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.700 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216861 s, 0.0 kB/s 2024-04-26T20:03:42.701 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.701 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.825 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261442 s, 0.0 kB/s 2024-04-26T20:03:42.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.884 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:42.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:42.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:42.947 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215224 s, 0.0 kB/s 2024-04-26T20:03:42.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:42.948 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:42.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:42.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255789 s, 0.0 kB/s 2024-04-26T20:03:43.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.113 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.167 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.230 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000317617 s, 0.0 kB/s 2024-04-26T20:03:43.231 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.231 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.284 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165414 s, 0.0 kB/s 2024-04-26T20:03:43.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.401 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.515 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263078 s, 0.0 kB/s 2024-04-26T20:03:43.516 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.517 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.573 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.636 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271602 s, 0.0 kB/s 2024-04-26T20:03:43.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.638 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.692 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020213 s, 0.0 kB/s 2024-04-26T20:03:43.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.816 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:43.873 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192555 s, 0.0 kB/s 2024-04-26T20:03:43.874 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:43.874 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:43.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:43.924 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.035 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235331 s, 0.0 kB/s 2024-04-26T20:03:44.036 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.036 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.090 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.154 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.154 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.154 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245238 s, 0.0 kB/s 2024-04-26T20:03:44.155 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.155 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.208 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174209 s, 0.0 kB/s 2024-04-26T20:03:44.323 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.323 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.376 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.441 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.442 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175257 s, 0.0 kB/s 2024-04-26T20:03:44.443 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.443 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.492 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.606 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000290351 s, 0.0 kB/s 2024-04-26T20:03:44.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.608 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.664 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.730 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.730 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.730 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293381 s, 0.0 kB/s 2024-04-26T20:03:44.731 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.732 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021618 s, 0.0 kB/s 2024-04-26T20:03:44.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:44.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:44.904 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:44.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:44.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:44.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312822 s, 0.0 kB/s 2024-04-26T20:03:44.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:44.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019912 s, 0.0 kB/s 2024-04-26T20:03:45.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.145 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.209 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207572 s, 0.0 kB/s 2024-04-26T20:03:45.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.210 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.262 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000339175 s, 0.0 kB/s 2024-04-26T20:03:45.374 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.428 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.498 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280862 s, 0.0 kB/s 2024-04-26T20:03:45.499 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.499 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.619 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.619 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.619 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189014 s, 0.0 kB/s 2024-04-26T20:03:45.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.620 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.668 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.780 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.780 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.780 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172929 s, 0.0 kB/s 2024-04-26T20:03:45.781 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.837 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:45.899 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:45.899 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:45.899 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203925 s, 0.0 kB/s 2024-04-26T20:03:45.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:45.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:45.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:45.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.066 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000308064 s, 0.0 kB/s 2024-04-26T20:03:46.068 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.068 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.121 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156419 s, 0.0 kB/s 2024-04-26T20:03:46.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.314 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.314 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024112 s, 0.0 kB/s 2024-04-26T20:03:46.315 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.315 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.442 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000313698 s, 0.0 kB/s 2024-04-26T20:03:46.443 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.444 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307207 s, 0.0 kB/s 2024-04-26T20:03:46.563 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.563 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000326401 s, 0.0 kB/s 2024-04-26T20:03:46.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.743 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.812 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000301719 s, 0.0 kB/s 2024-04-26T20:03:46.813 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.814 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.868 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:46.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:46.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:46.934 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237844 s, 0.0 kB/s 2024-04-26T20:03:46.935 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:46.935 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:46.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:46.987 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000276865 s, 0.0 kB/s 2024-04-26T20:03:47.053 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.053 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.104 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222324 s, 0.0 kB/s 2024-04-26T20:03:47.167 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.167 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.217 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227045 s, 0.0 kB/s 2024-04-26T20:03:47.331 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.331 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000351937 s, 0.0 kB/s 2024-04-26T20:03:47.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.460 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.514 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.579 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185752 s, 0.0 kB/s 2024-04-26T20:03:47.580 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.580 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.633 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.699 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.700 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000321181 s, 0.0 kB/s 2024-04-26T20:03:47.701 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.701 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.828 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267162 s, 0.0 kB/s 2024-04-26T20:03:47.829 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.829 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:47.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:47.883 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:47.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:47.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:47.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211414 s, 0.0 kB/s 2024-04-26T20:03:47.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:47.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.006 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279415 s, 0.0 kB/s 2024-04-26T20:03:48.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.195 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000334167 s, 0.0 kB/s 2024-04-26T20:03:48.196 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315687 s, 0.0 kB/s 2024-04-26T20:03:48.318 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.319 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.482 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000384605 s, 0.0 kB/s 2024-04-26T20:03:48.483 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.535 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.644 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223685 s, 0.0 kB/s 2024-04-26T20:03:48.645 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.645 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.696 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.758 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228183 s, 0.0 kB/s 2024-04-26T20:03:48.759 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.760 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.816 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:48.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000309931 s, 0.0 kB/s 2024-04-26T20:03:48.886 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:48.886 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:48.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:48.940 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.000 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.000 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.000 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205188 s, 0.0 kB/s 2024-04-26T20:03:49.001 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.001 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.162 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.162 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.162 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180723 s, 0.0 kB/s 2024-04-26T20:03:49.163 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.163 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.270 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000136972 s, 0.0 kB/s 2024-04-26T20:03:49.271 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.271 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.321 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00033284 s, 0.0 kB/s 2024-04-26T20:03:49.437 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.438 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.492 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000299267 s, 0.0 kB/s 2024-04-26T20:03:49.555 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.556 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.611 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.677 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186973 s, 0.0 kB/s 2024-04-26T20:03:49.678 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.733 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.801 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166508 s, 0.0 kB/s 2024-04-26T20:03:49.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.803 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:49.918 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209349 s, 0.0 kB/s 2024-04-26T20:03:49.919 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:49.919 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:49.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:49.974 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.040 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181685 s, 0.0 kB/s 2024-04-26T20:03:50.041 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.042 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.094 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.157 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237398 s, 0.0 kB/s 2024-04-26T20:03:50.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.158 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.211 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.277 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.277 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.277 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230803 s, 0.0 kB/s 2024-04-26T20:03:50.279 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.279 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.395 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172289 s, 0.0 kB/s 2024-04-26T20:03:50.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.449 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.512 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171625 s, 0.0 kB/s 2024-04-26T20:03:50.513 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.564 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000298838 s, 0.0 kB/s 2024-04-26T20:03:50.635 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.692 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.757 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272552 s, 0.0 kB/s 2024-04-26T20:03:50.758 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.758 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.810 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:50.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:50.920 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:50.920 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246031 s, 0.0 kB/s 2024-04-26T20:03:50.921 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:50.921 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:50.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:50.975 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.040 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147506 s, 0.0 kB/s 2024-04-26T20:03:51.042 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.042 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.092 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.199 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000296174 s, 0.0 kB/s 2024-04-26T20:03:51.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.200 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.360 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196266 s, 0.0 kB/s 2024-04-26T20:03:51.361 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.362 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.416 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.485 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213505 s, 0.0 kB/s 2024-04-26T20:03:51.487 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.487 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.540 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.605 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000288909 s, 0.0 kB/s 2024-04-26T20:03:51.606 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.606 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.659 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.728 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201646 s, 0.0 kB/s 2024-04-26T20:03:51.729 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.730 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.779 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:51.886 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:51.887 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:51.887 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229227 s, 0.0 kB/s 2024-04-26T20:03:51.888 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:51.888 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:51.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:51.945 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292459 s, 0.0 kB/s 2024-04-26T20:03:52.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.013 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.069 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.136 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.136 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.136 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230563 s, 0.0 kB/s 2024-04-26T20:03:52.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.193 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.265 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294733 s, 0.0 kB/s 2024-04-26T20:03:52.267 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.267 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.322 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.385 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.385 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.385 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202486 s, 0.0 kB/s 2024-04-26T20:03:52.387 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.387 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.439 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.505 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185578 s, 0.0 kB/s 2024-04-26T20:03:52.506 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.506 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.630 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.630 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277823 s, 0.0 kB/s 2024-04-26T20:03:52.631 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.631 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.684 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.750 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259183 s, 0.0 kB/s 2024-04-26T20:03:52.751 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.751 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.807 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.872 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.873 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258254 s, 0.0 kB/s 2024-04-26T20:03:52.874 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.875 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:52.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:52.929 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:52.997 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:52.997 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:52.997 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155721 s, 0.0 kB/s 2024-04-26T20:03:52.998 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:52.998 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.121 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030303 s, 0.0 kB/s 2024-04-26T20:03:53.122 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.123 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.177 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.238 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168744 s, 0.0 kB/s 2024-04-26T20:03:53.239 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.239 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.293 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234426 s, 0.0 kB/s 2024-04-26T20:03:53.369 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.369 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.426 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000276547 s, 0.0 kB/s 2024-04-26T20:03:53.495 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.495 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.550 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026109 s, 0.0 kB/s 2024-04-26T20:03:53.621 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.622 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.677 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.742 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267577 s, 0.0 kB/s 2024-04-26T20:03:53.743 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.743 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154566 s, 0.0 kB/s 2024-04-26T20:03:53.854 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.855 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:53.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:53.907 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:53.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:53.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:53.974 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200624 s, 0.0 kB/s 2024-04-26T20:03:53.975 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:53.975 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270426 s, 0.0 kB/s 2024-04-26T20:03:54.096 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.150 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.217 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243194 s, 0.0 kB/s 2024-04-26T20:03:54.218 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.218 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.273 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273979 s, 0.0 kB/s 2024-04-26T20:03:54.345 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.399 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.467 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000325004 s, 0.0 kB/s 2024-04-26T20:03:54.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.522 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.589 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180677 s, 0.0 kB/s 2024-04-26T20:03:54.591 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.591 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.646 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304904 s, 0.0 kB/s 2024-04-26T20:03:54.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.719 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.776 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.840 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.840 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.840 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248039 s, 0.0 kB/s 2024-04-26T20:03:54.842 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.842 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:54.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:54.899 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:54.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:54.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:54.966 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256308 s, 0.0 kB/s 2024-04-26T20:03:54.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:54.967 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.089 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.089 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.089 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244832 s, 0.0 kB/s 2024-04-26T20:03:55.090 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.090 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.146 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000299823 s, 0.0 kB/s 2024-04-26T20:03:55.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.211 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.265 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.327 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.327 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.327 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201193 s, 0.0 kB/s 2024-04-26T20:03:55.328 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.328 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.383 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000338934 s, 0.0 kB/s 2024-04-26T20:03:55.456 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.456 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.514 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.581 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272884 s, 0.0 kB/s 2024-04-26T20:03:55.582 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.583 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.638 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.705 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190804 s, 0.0 kB/s 2024-04-26T20:03:55.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.763 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192887 s, 0.0 kB/s 2024-04-26T20:03:55.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.834 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:55.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:55.889 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:55.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:55.954 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282008 s, 0.0 kB/s 2024-04-26T20:03:55.955 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:55.955 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.012 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000354268 s, 0.0 kB/s 2024-04-26T20:03:56.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.136 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152338 s, 0.0 kB/s 2024-04-26T20:03:56.201 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.201 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.313 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146087 s, 0.0 kB/s 2024-04-26T20:03:56.314 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.314 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.367 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.431 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148405 s, 0.0 kB/s 2024-04-26T20:03:56.432 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.433 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.484 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.551 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249538 s, 0.0 kB/s 2024-04-26T20:03:56.553 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.607 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.668 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263961 s, 0.0 kB/s 2024-04-26T20:03:56.669 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.719 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.827 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263684 s, 0.0 kB/s 2024-04-26T20:03:56.828 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.828 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:56.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:56.883 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:56.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:56.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:56.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195208 s, 0.0 kB/s 2024-04-26T20:03:56.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:56.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.008 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198238 s, 0.0 kB/s 2024-04-26T20:03:57.075 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.189 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001946 s, 0.0 kB/s 2024-04-26T20:03:57.190 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.190 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.245 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.310 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213374 s, 0.0 kB/s 2024-04-26T20:03:57.311 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.455 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.525 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253804 s, 0.0 kB/s 2024-04-26T20:03:57.526 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.526 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.577 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.636 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203078 s, 0.0 kB/s 2024-04-26T20:03:57.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.637 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.689 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.753 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.753 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.753 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214452 s, 0.0 kB/s 2024-04-26T20:03:57.754 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.754 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.805 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:57.916 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:57.916 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:57.916 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244511 s, 0.0 kB/s 2024-04-26T20:03:57.917 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:57.917 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:57.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:57.972 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024017 s, 0.0 kB/s 2024-04-26T20:03:58.038 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.038 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.091 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200259 s, 0.0 kB/s 2024-04-26T20:03:58.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.159 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.213 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208425 s, 0.0 kB/s 2024-04-26T20:03:58.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.278 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.332 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000338447 s, 0.0 kB/s 2024-04-26T20:03:58.403 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.403 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.460 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.527 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.527 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.528 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228252 s, 0.0 kB/s 2024-04-26T20:03:58.529 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.529 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.584 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.650 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197761 s, 0.0 kB/s 2024-04-26T20:03:58.651 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.704 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.765 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.766 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245961 s, 0.0 kB/s 2024-04-26T20:03:58.767 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.767 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.820 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.882 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.883 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.883 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229226 s, 0.0 kB/s 2024-04-26T20:03:58.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:58.884 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:58.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:58.938 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:58.999 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291352 s, 0.0 kB/s 2024-04-26T20:03:59.001 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.001 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.056 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.122 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182672 s, 0.0 kB/s 2024-04-26T20:03:59.123 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.123 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.234 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262275 s, 0.0 kB/s 2024-04-26T20:03:59.236 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.236 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.286 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220527 s, 0.0 kB/s 2024-04-26T20:03:59.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173857 s, 0.0 kB/s 2024-04-26T20:03:59.475 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.475 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.530 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.599 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.599 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.599 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190722 s, 0.0 kB/s 2024-04-26T20:03:59.600 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.600 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.654 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.718 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248385 s, 0.0 kB/s 2024-04-26T20:03:59.719 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.719 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.774 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.842 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188596 s, 0.0 kB/s 2024-04-26T20:03:59.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:03:59.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:03:59.897 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:03:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:03:59.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:03:59.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253122 s, 0.0 kB/s 2024-04-26T20:03:59.968 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:03:59.968 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.086 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192276 s, 0.0 kB/s 2024-04-26T20:04:00.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.087 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.137 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.248 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221711 s, 0.0 kB/s 2024-04-26T20:04:00.250 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.250 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.306 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.379 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021203 s, 0.0 kB/s 2024-04-26T20:04:00.380 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.435 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.506 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260503 s, 0.0 kB/s 2024-04-26T20:04:00.508 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.508 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.563 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.634 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315727 s, 0.0 kB/s 2024-04-26T20:04:00.635 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.692 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025101 s, 0.0 kB/s 2024-04-26T20:04:00.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.818 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:00.882 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:00.883 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:00.883 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198012 s, 0.0 kB/s 2024-04-26T20:04:00.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:00.884 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:00.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:00.937 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.004 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212712 s, 0.0 kB/s 2024-04-26T20:04:01.005 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.005 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.061 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000476381 s, 0.0 kB/s 2024-04-26T20:04:01.139 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.139 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.263 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174853 s, 0.0 kB/s 2024-04-26T20:04:01.264 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.264 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.322 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000310189 s, 0.0 kB/s 2024-04-26T20:04:01.398 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222009 s, 0.0 kB/s 2024-04-26T20:04:01.527 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.527 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.579 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.649 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274723 s, 0.0 kB/s 2024-04-26T20:04:01.650 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.650 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.704 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205665 s, 0.0 kB/s 2024-04-26T20:04:01.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:01.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:01.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:01.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282318 s, 0.0 kB/s 2024-04-26T20:04:01.903 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:01.903 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:01.959 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.028 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212219 s, 0.0 kB/s 2024-04-26T20:04:02.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.087 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267653 s, 0.0 kB/s 2024-04-26T20:04:02.156 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.156 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.207 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.321 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.322 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002552 s, 0.0 kB/s 2024-04-26T20:04:02.323 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.323 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.374 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245885 s, 0.0 kB/s 2024-04-26T20:04:02.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.504 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260684 s, 0.0 kB/s 2024-04-26T20:04:02.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.625 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.692 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216023 s, 0.0 kB/s 2024-04-26T20:04:02.694 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.744 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000352124 s, 0.0 kB/s 2024-04-26T20:04:02.812 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.812 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.864 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:02.935 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214035 s, 0.0 kB/s 2024-04-26T20:04:02.936 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:02.936 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:02.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:02.993 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.062 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000314676 s, 0.0 kB/s 2024-04-26T20:04:03.063 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.063 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.112 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218949 s, 0.0 kB/s 2024-04-26T20:04:03.226 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.227 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.276 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.388 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227494 s, 0.0 kB/s 2024-04-26T20:04:03.390 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.390 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.444 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259953 s, 0.0 kB/s 2024-04-26T20:04:03.511 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.511 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.563 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286088 s, 0.0 kB/s 2024-04-26T20:04:03.634 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.634 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258975 s, 0.0 kB/s 2024-04-26T20:04:03.751 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.752 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.802 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:03.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222786 s, 0.0 kB/s 2024-04-26T20:04:03.908 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:03.908 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:03.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:03.957 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000276315 s, 0.0 kB/s 2024-04-26T20:04:04.070 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.070 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.121 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.234 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280152 s, 0.0 kB/s 2024-04-26T20:04:04.235 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.235 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.290 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.358 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000328111 s, 0.0 kB/s 2024-04-26T20:04:04.359 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.477 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.478 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.478 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211227 s, 0.0 kB/s 2024-04-26T20:04:04.479 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.479 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182437 s, 0.0 kB/s 2024-04-26T20:04:04.597 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.597 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.647 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.710 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.710 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.710 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000281769 s, 0.0 kB/s 2024-04-26T20:04:04.711 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.767 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277646 s, 0.0 kB/s 2024-04-26T20:04:04.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:04.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:04.892 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:04.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:04.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:04.959 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304715 s, 0.0 kB/s 2024-04-26T20:04:04.960 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:04.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177415 s, 0.0 kB/s 2024-04-26T20:04:05.083 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.083 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.137 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.204 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207159 s, 0.0 kB/s 2024-04-26T20:04:05.207 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.207 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.257 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168879 s, 0.0 kB/s 2024-04-26T20:04:05.318 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.319 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.372 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.438 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230509 s, 0.0 kB/s 2024-04-26T20:04:05.440 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.440 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.495 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.559 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186944 s, 0.0 kB/s 2024-04-26T20:04:05.560 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.560 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.615 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.682 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237276 s, 0.0 kB/s 2024-04-26T20:04:05.683 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.683 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210496 s, 0.0 kB/s 2024-04-26T20:04:05.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.803 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.859 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:05.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:05.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:05.930 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207698 s, 0.0 kB/s 2024-04-26T20:04:05.931 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:05.931 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:05.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:05.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.047 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241002 s, 0.0 kB/s 2024-04-26T20:04:06.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.049 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.104 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173674 s, 0.0 kB/s 2024-04-26T20:04:06.174 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.174 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.293 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021822 s, 0.0 kB/s 2024-04-26T20:04:06.294 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.294 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.348 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.411 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.411 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.411 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257342 s, 0.0 kB/s 2024-04-26T20:04:06.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.412 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.464 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249175 s, 0.0 kB/s 2024-04-26T20:04:06.529 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.530 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.583 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.652 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.653 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.653 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025476 s, 0.0 kB/s 2024-04-26T20:04:06.654 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.654 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.709 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.772 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026857 s, 0.0 kB/s 2024-04-26T20:04:06.773 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.773 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.828 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:06.894 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:06.894 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:06.894 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155153 s, 0.0 kB/s 2024-04-26T20:04:06.895 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:06.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:06.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:06.950 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.022 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000295209 s, 0.0 kB/s 2024-04-26T20:04:07.023 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.023 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.079 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.143 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148776 s, 0.0 kB/s 2024-04-26T20:04:07.144 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.144 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.197 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.257 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165717 s, 0.0 kB/s 2024-04-26T20:04:07.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.258 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.312 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.434 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000348184 s, 0.0 kB/s 2024-04-26T20:04:07.436 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.436 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.494 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.569 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.569 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.569 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014724 s, 0.0 kB/s 2024-04-26T20:04:07.570 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.570 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.626 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.693 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.694 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242917 s, 0.0 kB/s 2024-04-26T20:04:07.695 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.695 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188958 s, 0.0 kB/s 2024-04-26T20:04:07.817 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.817 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.872 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:07.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:07.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:07.937 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171076 s, 0.0 kB/s 2024-04-26T20:04:07.938 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:07.939 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:07.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:07.989 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.054 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.054 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168054 s, 0.0 kB/s 2024-04-26T20:04:08.055 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.055 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.174 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187189 s, 0.0 kB/s 2024-04-26T20:04:08.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.175 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.230 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.297 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.297 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.297 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224093 s, 0.0 kB/s 2024-04-26T20:04:08.298 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.298 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.352 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.414 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206926 s, 0.0 kB/s 2024-04-26T20:04:08.415 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.415 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.466 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274288 s, 0.0 kB/s 2024-04-26T20:04:08.585 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.585 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.696 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022195 s, 0.0 kB/s 2024-04-26T20:04:08.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.697 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.750 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193325 s, 0.0 kB/s 2024-04-26T20:04:08.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.811 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:08.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:08.862 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:08.970 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:08.970 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:08.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000289833 s, 0.0 kB/s 2024-04-26T20:04:08.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:08.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.026 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.092 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199938 s, 0.0 kB/s 2024-04-26T20:04:09.093 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.094 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184028 s, 0.0 kB/s 2024-04-26T20:04:09.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.302 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.368 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000406976 s, 0.0 kB/s 2024-04-26T20:04:09.370 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.370 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.425 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.490 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.490 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.490 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000300687 s, 0.0 kB/s 2024-04-26T20:04:09.492 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.492 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.546 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.612 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.612 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.612 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223784 s, 0.0 kB/s 2024-04-26T20:04:09.613 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.614 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.671 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230157 s, 0.0 kB/s 2024-04-26T20:04:09.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.788 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.856 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196525 s, 0.0 kB/s 2024-04-26T20:04:09.857 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.857 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:09.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:09.913 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:09.981 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:09.981 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:09.981 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243892 s, 0.0 kB/s 2024-04-26T20:04:09.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:09.982 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.036 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.103 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.103 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218448 s, 0.0 kB/s 2024-04-26T20:04:10.104 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.104 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.160 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.221 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022273 s, 0.0 kB/s 2024-04-26T20:04:10.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.222 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.276 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.337 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.337 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.337 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204095 s, 0.0 kB/s 2024-04-26T20:04:10.338 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.338 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.387 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:04:10.479 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:04:10.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.497 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280979 s, 0.0 kB/s 2024-04-26T20:04:10.499 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.499 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.588 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.699 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.699 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.699 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257792 s, 0.0 kB/s 2024-04-26T20:04:10.700 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.701 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.821 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018393 s, 0.0 kB/s 2024-04-26T20:04:10.822 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.876 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:10.938 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:10.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:10.939 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222343 s, 0.0 kB/s 2024-04-26T20:04:10.940 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:10.940 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:10.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:10.991 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165937 s, 0.0 kB/s 2024-04-26T20:04:11.103 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.103 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.154 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.215 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.216 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197677 s, 0.0 kB/s 2024-04-26T20:04:11.217 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.217 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.269 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.379 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158234 s, 0.0 kB/s 2024-04-26T20:04:11.380 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277926 s, 0.0 kB/s 2024-04-26T20:04:11.499 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.499 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.627 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024866 s, 0.0 kB/s 2024-04-26T20:04:11.628 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.628 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.749 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.749 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217733 s, 0.0 kB/s 2024-04-26T20:04:11.750 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.750 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.864 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.864 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.865 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293231 s, 0.0 kB/s 2024-04-26T20:04:11.866 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.866 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:11.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:11.921 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:11.987 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:11.987 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:11.987 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196329 s, 0.0 kB/s 2024-04-26T20:04:11.988 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:11.988 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.039 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.104 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.104 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.104 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209932 s, 0.0 kB/s 2024-04-26T20:04:12.105 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.105 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.156 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209624 s, 0.0 kB/s 2024-04-26T20:04:12.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.223 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231069 s, 0.0 kB/s 2024-04-26T20:04:12.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.381 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.434 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.500 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.500 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000313831 s, 0.0 kB/s 2024-04-26T20:04:12.501 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.556 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.623 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.623 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.623 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160594 s, 0.0 kB/s 2024-04-26T20:04:12.624 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.624 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.678 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.742 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246051 s, 0.0 kB/s 2024-04-26T20:04:12.743 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.743 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:12.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:12.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:12.859 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293436 s, 0.0 kB/s 2024-04-26T20:04:12.860 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:12.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:12.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:12.912 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.022 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259107 s, 0.0 kB/s 2024-04-26T20:04:13.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.024 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.080 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.144 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.145 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250258 s, 0.0 kB/s 2024-04-26T20:04:13.146 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.146 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.201 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.271 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.271 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000368069 s, 0.0 kB/s 2024-04-26T20:04:13.272 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.272 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.394 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152224 s, 0.0 kB/s 2024-04-26T20:04:13.395 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.447 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.513 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252306 s, 0.0 kB/s 2024-04-26T20:04:13.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.569 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179445 s, 0.0 kB/s 2024-04-26T20:04:13.634 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.634 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.760 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180911 s, 0.0 kB/s 2024-04-26T20:04:13.761 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.761 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.814 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258801 s, 0.0 kB/s 2024-04-26T20:04:13.878 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.878 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:13.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:13.931 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:13.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:13.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:13.993 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175449 s, 0.0 kB/s 2024-04-26T20:04:13.995 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:13.995 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.045 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.110 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000301318 s, 0.0 kB/s 2024-04-26T20:04:14.112 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.112 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.162 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230873 s, 0.0 kB/s 2024-04-26T20:04:14.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.321 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.385 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.385 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.385 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315902 s, 0.0 kB/s 2024-04-26T20:04:14.387 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.387 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.443 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.508 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209396 s, 0.0 kB/s 2024-04-26T20:04:14.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.509 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.564 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.634 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188645 s, 0.0 kB/s 2024-04-26T20:04:14.635 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.689 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260503 s, 0.0 kB/s 2024-04-26T20:04:14.753 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.753 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.804 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291363 s, 0.0 kB/s 2024-04-26T20:04:14.916 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:14.916 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:14.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:14.971 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.036 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248571 s, 0.0 kB/s 2024-04-26T20:04:15.037 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.037 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.090 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.158 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00031345 s, 0.0 kB/s 2024-04-26T20:04:15.159 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.159 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.280 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225461 s, 0.0 kB/s 2024-04-26T20:04:15.280 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.281 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.403 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247254 s, 0.0 kB/s 2024-04-26T20:04:15.405 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.405 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.462 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218495 s, 0.0 kB/s 2024-04-26T20:04:15.534 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.534 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.588 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.656 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176328 s, 0.0 kB/s 2024-04-26T20:04:15.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.657 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.711 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.774 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239379 s, 0.0 kB/s 2024-04-26T20:04:15.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.828 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:15.897 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:15.897 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:15.897 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189108 s, 0.0 kB/s 2024-04-26T20:04:15.898 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:15.898 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:15.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:15.952 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206426 s, 0.0 kB/s 2024-04-26T20:04:16.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213975 s, 0.0 kB/s 2024-04-26T20:04:16.130 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.130 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.184 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.244 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210261 s, 0.0 kB/s 2024-04-26T20:04:16.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.245 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.365 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.365 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202899 s, 0.0 kB/s 2024-04-26T20:04:16.366 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.366 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.420 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.485 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241087 s, 0.0 kB/s 2024-04-26T20:04:16.486 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.487 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.540 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.607 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.607 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.607 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025387 s, 0.0 kB/s 2024-04-26T20:04:16.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.608 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000289041 s, 0.0 kB/s 2024-04-26T20:04:16.725 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.725 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.778 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.842 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171163 s, 0.0 kB/s 2024-04-26T20:04:16.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:16.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:16.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:16.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:16.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:16.959 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214471 s, 0.0 kB/s 2024-04-26T20:04:16.960 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:16.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.015 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196196 s, 0.0 kB/s 2024-04-26T20:04:17.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.131 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270681 s, 0.0 kB/s 2024-04-26T20:04:17.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.245 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240805 s, 0.0 kB/s 2024-04-26T20:04:17.358 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.358 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.409 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.471 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.472 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019656 s, 0.0 kB/s 2024-04-26T20:04:17.473 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.473 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.527 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.590 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198134 s, 0.0 kB/s 2024-04-26T20:04:17.591 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.591 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.640 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234601 s, 0.0 kB/s 2024-04-26T20:04:17.701 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.701 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.760 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:17.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:17.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:17.914 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260824 s, 0.0 kB/s 2024-04-26T20:04:17.915 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:17.915 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:17.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:17.974 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274783 s, 0.0 kB/s 2024-04-26T20:04:18.046 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.046 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.102 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.173 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.173 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.173 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279496 s, 0.0 kB/s 2024-04-26T20:04:18.174 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.174 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.229 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.293 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209239 s, 0.0 kB/s 2024-04-26T20:04:18.294 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.294 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.343 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.457 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205506 s, 0.0 kB/s 2024-04-26T20:04:18.459 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.459 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.513 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.576 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236499 s, 0.0 kB/s 2024-04-26T20:04:18.577 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.577 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022586 s, 0.0 kB/s 2024-04-26T20:04:18.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.741 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.793 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:18.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:18.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:18.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261201 s, 0.0 kB/s 2024-04-26T20:04:18.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:18.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:18.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:18.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238368 s, 0.0 kB/s 2024-04-26T20:04:19.023 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.023 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.075 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191084 s, 0.0 kB/s 2024-04-26T20:04:19.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.199 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.261 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.261 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.261 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000141268 s, 0.0 kB/s 2024-04-26T20:04:19.262 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.262 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.311 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.422 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196434 s, 0.0 kB/s 2024-04-26T20:04:19.423 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.424 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.480 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194178 s, 0.0 kB/s 2024-04-26T20:04:19.548 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.548 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.605 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.670 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.671 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172275 s, 0.0 kB/s 2024-04-26T20:04:19.672 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.672 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.728 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.791 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.791 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.791 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211457 s, 0.0 kB/s 2024-04-26T20:04:19.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.793 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.845 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:19.910 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:19.910 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:19.910 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238314 s, 0.0 kB/s 2024-04-26T20:04:19.912 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:19.912 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:19.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:19.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.026 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144661 s, 0.0 kB/s 2024-04-26T20:04:20.027 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.078 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.188 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243609 s, 0.0 kB/s 2024-04-26T20:04:20.189 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.243 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.305 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200035 s, 0.0 kB/s 2024-04-26T20:04:20.306 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.306 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.359 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.422 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287444 s, 0.0 kB/s 2024-04-26T20:04:20.424 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.424 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180482 s, 0.0 kB/s 2024-04-26T20:04:20.548 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.548 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.666 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242428 s, 0.0 kB/s 2024-04-26T20:04:20.667 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.667 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.722 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.787 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150744 s, 0.0 kB/s 2024-04-26T20:04:20.788 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.843 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202883 s, 0.0 kB/s 2024-04-26T20:04:20.910 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:20.910 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:20.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:20.965 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.031 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000329296 s, 0.0 kB/s 2024-04-26T20:04:21.033 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.033 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.082 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.194 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253262 s, 0.0 kB/s 2024-04-26T20:04:21.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.249 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.305 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138161 s, 0.0 kB/s 2024-04-26T20:04:21.306 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.307 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.356 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.470 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231231 s, 0.0 kB/s 2024-04-26T20:04:21.471 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.471 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.595 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.595 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.595 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000297787 s, 0.0 kB/s 2024-04-26T20:04:21.597 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.597 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000297711 s, 0.0 kB/s 2024-04-26T20:04:21.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.718 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.772 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.840 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237315 s, 0.0 kB/s 2024-04-26T20:04:21.842 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.842 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:21.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:21.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000344507 s, 0.0 kB/s 2024-04-26T20:04:21.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:21.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306981 s, 0.0 kB/s 2024-04-26T20:04:22.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.134 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237419 s, 0.0 kB/s 2024-04-26T20:04:22.194 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.194 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.248 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.313 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180223 s, 0.0 kB/s 2024-04-26T20:04:22.314 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.314 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.366 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.429 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.429 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.429 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275103 s, 0.0 kB/s 2024-04-26T20:04:22.431 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.431 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.555 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.555 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207843 s, 0.0 kB/s 2024-04-26T20:04:22.556 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.556 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.612 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315265 s, 0.0 kB/s 2024-04-26T20:04:22.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.681 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.797 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210864 s, 0.0 kB/s 2024-04-26T20:04:22.799 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.799 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.849 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000347253 s, 0.0 kB/s 2024-04-26T20:04:22.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:22.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:22.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:22.967 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.030 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.030 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.030 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190389 s, 0.0 kB/s 2024-04-26T20:04:23.031 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.031 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.085 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.149 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235803 s, 0.0 kB/s 2024-04-26T20:04:23.150 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.151 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.268 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014482 s, 0.0 kB/s 2024-04-26T20:04:23.269 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.269 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.319 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.382 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000127042 s, 0.0 kB/s 2024-04-26T20:04:23.383 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.545 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200764 s, 0.0 kB/s 2024-04-26T20:04:23.546 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.546 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.668 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253059 s, 0.0 kB/s 2024-04-26T20:04:23.669 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241071 s, 0.0 kB/s 2024-04-26T20:04:23.783 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.783 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:23.942 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:23.942 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:23.942 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189028 s, 0.0 kB/s 2024-04-26T20:04:23.943 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:23.943 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:23.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:23.996 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179773 s, 0.0 kB/s 2024-04-26T20:04:24.066 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.066 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.119 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218864 s, 0.0 kB/s 2024-04-26T20:04:24.233 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.233 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.287 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182084 s, 0.0 kB/s 2024-04-26T20:04:24.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218435 s, 0.0 kB/s 2024-04-26T20:04:24.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226972 s, 0.0 kB/s 2024-04-26T20:04:24.587 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.587 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225667 s, 0.0 kB/s 2024-04-26T20:04:24.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.759 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017892 s, 0.0 kB/s 2024-04-26T20:04:24.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.826 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.881 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:24.943 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:24.943 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:24.943 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237349 s, 0.0 kB/s 2024-04-26T20:04:24.944 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:24.944 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:24.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:24.997 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.067 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000326757 s, 0.0 kB/s 2024-04-26T20:04:25.068 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.068 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.116 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170811 s, 0.0 kB/s 2024-04-26T20:04:25.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.283 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292504 s, 0.0 kB/s 2024-04-26T20:04:25.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.351 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.476 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.476 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.477 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000141729 s, 0.0 kB/s 2024-04-26T20:04:25.478 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.478 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.639 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240698 s, 0.0 kB/s 2024-04-26T20:04:25.640 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.640 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.694 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.758 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294748 s, 0.0 kB/s 2024-04-26T20:04:25.760 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.760 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.811 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:25.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:25.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:25.923 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000324811 s, 0.0 kB/s 2024-04-26T20:04:25.924 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:25.925 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:25.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:25.980 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.044 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184964 s, 0.0 kB/s 2024-04-26T20:04:26.044 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.044 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230158 s, 0.0 kB/s 2024-04-26T20:04:26.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.162 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.215 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.279 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219576 s, 0.0 kB/s 2024-04-26T20:04:26.280 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.280 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.394 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259877 s, 0.0 kB/s 2024-04-26T20:04:26.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.516 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018709 s, 0.0 kB/s 2024-04-26T20:04:26.517 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.517 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.571 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.627 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000125613 s, 0.0 kB/s 2024-04-26T20:04:26.628 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.628 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258613 s, 0.0 kB/s 2024-04-26T20:04:26.787 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.787 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.844 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:26.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000355594 s, 0.0 kB/s 2024-04-26T20:04:26.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:26.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:26.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:26.969 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.032 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217728 s, 0.0 kB/s 2024-04-26T20:04:27.033 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.034 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.086 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.201 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234626 s, 0.0 kB/s 2024-04-26T20:04:27.202 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.366 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203419 s, 0.0 kB/s 2024-04-26T20:04:27.368 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.368 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.424 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000353224 s, 0.0 kB/s 2024-04-26T20:04:27.495 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.495 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.547 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.615 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183619 s, 0.0 kB/s 2024-04-26T20:04:27.616 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.616 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.667 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.729 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202632 s, 0.0 kB/s 2024-04-26T20:04:27.730 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.730 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.783 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:27.840 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138963 s, 0.0 kB/s 2024-04-26T20:04:27.842 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:27.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:27.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:27.895 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.002 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.002 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214323 s, 0.0 kB/s 2024-04-26T20:04:28.004 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.004 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166054 s, 0.0 kB/s 2024-04-26T20:04:28.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.160 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.211 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.314 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169042 s, 0.0 kB/s 2024-04-26T20:04:28.315 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.315 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.430 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268178 s, 0.0 kB/s 2024-04-26T20:04:28.431 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.432 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.485 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000337575 s, 0.0 kB/s 2024-04-26T20:04:28.549 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.549 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.663 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169875 s, 0.0 kB/s 2024-04-26T20:04:28.664 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.664 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.716 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244351 s, 0.0 kB/s 2024-04-26T20:04:28.781 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.834 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:28.891 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:28.891 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:28.892 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240884 s, 0.0 kB/s 2024-04-26T20:04:28.893 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:28.893 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:28.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:28.947 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.009 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.009 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.009 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265673 s, 0.0 kB/s 2024-04-26T20:04:29.010 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.010 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.064 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.128 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000440859 s, 0.0 kB/s 2024-04-26T20:04:29.129 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.180 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.238 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189356 s, 0.0 kB/s 2024-04-26T20:04:29.239 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.239 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.293 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020669 s, 0.0 kB/s 2024-04-26T20:04:29.355 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.355 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.408 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.469 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022084 s, 0.0 kB/s 2024-04-26T20:04:29.470 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.471 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.519 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187625 s, 0.0 kB/s 2024-04-26T20:04:29.627 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.627 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183485 s, 0.0 kB/s 2024-04-26T20:04:29.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.741 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167499 s, 0.0 kB/s 2024-04-26T20:04:29.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:29.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:29.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:29.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:29.965 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202441 s, 0.0 kB/s 2024-04-26T20:04:29.966 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:29.966 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.019 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016885 s, 0.0 kB/s 2024-04-26T20:04:30.075 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.123 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171468 s, 0.0 kB/s 2024-04-26T20:04:30.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.284 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.347 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178062 s, 0.0 kB/s 2024-04-26T20:04:30.348 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.398 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278962 s, 0.0 kB/s 2024-04-26T20:04:30.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.503 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.554 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.613 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172771 s, 0.0 kB/s 2024-04-26T20:04:30.614 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.614 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.670 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.732 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234927 s, 0.0 kB/s 2024-04-26T20:04:30.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.845 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000128818 s, 0.0 kB/s 2024-04-26T20:04:30.846 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.846 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:30.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:30.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:30.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:30.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:30.960 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245488 s, 0.0 kB/s 2024-04-26T20:04:30.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:30.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.079 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228947 s, 0.0 kB/s 2024-04-26T20:04:31.080 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.080 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.132 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195544 s, 0.0 kB/s 2024-04-26T20:04:31.188 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.188 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.239 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.298 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152555 s, 0.0 kB/s 2024-04-26T20:04:31.299 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.299 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.349 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.404 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138398 s, 0.0 kB/s 2024-04-26T20:04:31.405 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.405 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.455 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221787 s, 0.0 kB/s 2024-04-26T20:04:31.557 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.557 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.607 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.713 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260043 s, 0.0 kB/s 2024-04-26T20:04:31.714 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.715 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.768 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.872 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.872 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.872 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264601 s, 0.0 kB/s 2024-04-26T20:04:31.873 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.873 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:31.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:31.927 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185625 s, 0.0 kB/s 2024-04-26T20:04:31.986 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:31.986 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.036 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223493 s, 0.0 kB/s 2024-04-26T20:04:32.097 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.145 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014995 s, 0.0 kB/s 2024-04-26T20:04:32.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.304 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.368 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203671 s, 0.0 kB/s 2024-04-26T20:04:32.369 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.369 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.421 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.479 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.479 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.479 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160071 s, 0.0 kB/s 2024-04-26T20:04:32.480 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.480 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.628 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160627 s, 0.0 kB/s 2024-04-26T20:04:32.629 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.629 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.681 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.744 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.745 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.745 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024358 s, 0.0 kB/s 2024-04-26T20:04:32.746 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.796 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:32.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:32.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:32.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000283226 s, 0.0 kB/s 2024-04-26T20:04:32.902 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:32.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:32.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:32.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.014 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.014 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.014 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253208 s, 0.0 kB/s 2024-04-26T20:04:33.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.067 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175374 s, 0.0 kB/s 2024-04-26T20:04:33.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.127 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.183 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203834 s, 0.0 kB/s 2024-04-26T20:04:33.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.245 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.298 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144521 s, 0.0 kB/s 2024-04-26T20:04:33.358 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.358 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.407 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.465 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201123 s, 0.0 kB/s 2024-04-26T20:04:33.466 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.466 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.519 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.581 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168817 s, 0.0 kB/s 2024-04-26T20:04:33.582 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.582 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.634 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.696 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193326 s, 0.0 kB/s 2024-04-26T20:04:33.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.698 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.755 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.814 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.814 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.814 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151631 s, 0.0 kB/s 2024-04-26T20:04:33.815 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.815 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:33.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:33.865 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:33.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:33.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:33.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019116 s, 0.0 kB/s 2024-04-26T20:04:33.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:33.971 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.083 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.083 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.083 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206923 s, 0.0 kB/s 2024-04-26T20:04:34.084 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.085 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.140 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234155 s, 0.0 kB/s 2024-04-26T20:04:34.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.200 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.249 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.305 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159697 s, 0.0 kB/s 2024-04-26T20:04:34.306 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.306 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.425 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229744 s, 0.0 kB/s 2024-04-26T20:04:34.426 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.426 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.480 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.538 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180175 s, 0.0 kB/s 2024-04-26T20:04:34.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.540 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.590 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.675 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.676 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000140704 s, 0.0 kB/s 2024-04-26T20:04:34.677 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.677 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.773 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259801 s, 0.0 kB/s 2024-04-26T20:04:34.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:34.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:34.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:34.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:34.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:34.991 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217534 s, 0.0 kB/s 2024-04-26T20:04:34.992 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:34.992 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.042 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.097 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224833 s, 0.0 kB/s 2024-04-26T20:04:35.098 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.098 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154087 s, 0.0 kB/s 2024-04-26T20:04:35.247 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.247 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.294 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.398 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.398 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219767 s, 0.0 kB/s 2024-04-26T20:04:35.399 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000351957 s, 0.0 kB/s 2024-04-26T20:04:35.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.672 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.672 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.672 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237064 s, 0.0 kB/s 2024-04-26T20:04:35.673 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.673 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.727 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.787 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.787 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182531 s, 0.0 kB/s 2024-04-26T20:04:35.788 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.837 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:35.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:35.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:35.940 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022686 s, 0.0 kB/s 2024-04-26T20:04:35.941 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:35.941 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:35.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:35.995 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174828 s, 0.0 kB/s 2024-04-26T20:04:36.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.055 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.214 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218801 s, 0.0 kB/s 2024-04-26T20:04:36.216 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.216 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.269 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.324 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.324 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.324 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174641 s, 0.0 kB/s 2024-04-26T20:04:36.326 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.326 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.377 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.440 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186432 s, 0.0 kB/s 2024-04-26T20:04:36.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.494 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.554 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190195 s, 0.0 kB/s 2024-04-26T20:04:36.555 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.555 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.605 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225059 s, 0.0 kB/s 2024-04-26T20:04:36.714 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.714 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.830 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216218 s, 0.0 kB/s 2024-04-26T20:04:36.831 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.832 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.887 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:36.945 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:36.945 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:36.945 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175599 s, 0.0 kB/s 2024-04-26T20:04:36.947 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:36.947 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:36.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:36.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.060 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.060 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026269 s, 0.0 kB/s 2024-04-26T20:04:37.061 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.061 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.116 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.175 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247968 s, 0.0 kB/s 2024-04-26T20:04:37.179 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.232 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.293 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208036 s, 0.0 kB/s 2024-04-26T20:04:37.294 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.294 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.349 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.409 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243368 s, 0.0 kB/s 2024-04-26T20:04:37.410 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.410 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.464 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.526 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270841 s, 0.0 kB/s 2024-04-26T20:04:37.527 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.527 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.578 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00013999 s, 0.0 kB/s 2024-04-26T20:04:37.633 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.633 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194345 s, 0.0 kB/s 2024-04-26T20:04:37.745 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.745 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.859 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148983 s, 0.0 kB/s 2024-04-26T20:04:37.860 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:37.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:37.913 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:37.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:37.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:37.975 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187037 s, 0.0 kB/s 2024-04-26T20:04:37.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:37.976 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017951 s, 0.0 kB/s 2024-04-26T20:04:38.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.087 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.140 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.198 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199657 s, 0.0 kB/s 2024-04-26T20:04:38.199 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.312 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272889 s, 0.0 kB/s 2024-04-26T20:04:38.313 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.313 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.360 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286783 s, 0.0 kB/s 2024-04-26T20:04:38.467 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.467 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.518 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.626 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229796 s, 0.0 kB/s 2024-04-26T20:04:38.627 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.627 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.737 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.737 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.737 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203814 s, 0.0 kB/s 2024-04-26T20:04:38.738 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.738 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.888 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187507 s, 0.0 kB/s 2024-04-26T20:04:38.889 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.889 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:38.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:38.941 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:38.997 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:38.997 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:38.997 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166932 s, 0.0 kB/s 2024-04-26T20:04:38.998 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:38.999 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.049 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.107 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.107 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.108 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172178 s, 0.0 kB/s 2024-04-26T20:04:39.108 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.109 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.161 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.266 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292866 s, 0.0 kB/s 2024-04-26T20:04:39.267 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.378 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182102 s, 0.0 kB/s 2024-04-26T20:04:39.379 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.379 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.493 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019127 s, 0.0 kB/s 2024-04-26T20:04:39.494 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.494 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.548 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.605 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00013497 s, 0.0 kB/s 2024-04-26T20:04:39.606 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.607 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.657 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.761 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192939 s, 0.0 kB/s 2024-04-26T20:04:39.762 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.762 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.813 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.874 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.874 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.874 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260441 s, 0.0 kB/s 2024-04-26T20:04:39.876 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.876 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:39.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:39.929 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:39.988 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:39.988 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:39.988 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182855 s, 0.0 kB/s 2024-04-26T20:04:39.989 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:39.990 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.041 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019608 s, 0.0 kB/s 2024-04-26T20:04:40.100 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.100 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.147 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223633 s, 0.0 kB/s 2024-04-26T20:04:40.253 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.254 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.305 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.364 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269715 s, 0.0 kB/s 2024-04-26T20:04:40.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.365 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.420 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.480 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.480 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172179 s, 0.0 kB/s 2024-04-26T20:04:40.481 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.481 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.531 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:04:40.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:04:40.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265331 s, 0.0 kB/s 2024-04-26T20:04:40.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.814 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.815 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185121 s, 0.0 kB/s 2024-04-26T20:04:40.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.816 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:40.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:40.864 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:40.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:40.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:40.968 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262396 s, 0.0 kB/s 2024-04-26T20:04:40.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:40.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.021 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.079 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190108 s, 0.0 kB/s 2024-04-26T20:04:41.080 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.080 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.131 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138433 s, 0.0 kB/s 2024-04-26T20:04:41.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.276 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147735 s, 0.0 kB/s 2024-04-26T20:04:41.374 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.419 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.517 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.517 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.517 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018289 s, 0.0 kB/s 2024-04-26T20:04:41.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186381 s, 0.0 kB/s 2024-04-26T20:04:41.661 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.661 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.700 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.790 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.790 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.790 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000136807 s, 0.0 kB/s 2024-04-26T20:04:41.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.791 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172821 s, 0.0 kB/s 2024-04-26T20:04:41.931 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:41.931 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:41.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:41.973 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210514 s, 0.0 kB/s 2024-04-26T20:04:42.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.114 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160193 s, 0.0 kB/s 2024-04-26T20:04:42.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.210 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.252 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195988 s, 0.0 kB/s 2024-04-26T20:04:42.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.398 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.495 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.495 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161329 s, 0.0 kB/s 2024-04-26T20:04:42.496 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.496 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.638 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014189 s, 0.0 kB/s 2024-04-26T20:04:42.639 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.639 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.686 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.784 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002208 s, 0.0 kB/s 2024-04-26T20:04:42.785 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.785 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:42.938 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:42.938 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:42.938 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002225 s, 0.0 kB/s 2024-04-26T20:04:42.939 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:42.939 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:42.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.048 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149441 s, 0.0 kB/s 2024-04-26T20:04:43.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.049 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.101 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166554 s, 0.0 kB/s 2024-04-26T20:04:43.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.158 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151039 s, 0.0 kB/s 2024-04-26T20:04:43.302 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.302 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.350 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.458 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174178 s, 0.0 kB/s 2024-04-26T20:04:43.459 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.459 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.507 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.610 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.610 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.610 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000361722 s, 0.0 kB/s 2024-04-26T20:04:43.611 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.611 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184771 s, 0.0 kB/s 2024-04-26T20:04:43.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.814 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:43.868 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154899 s, 0.0 kB/s 2024-04-26T20:04:43.869 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:43.869 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:43.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:43.915 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.021 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206639 s, 0.0 kB/s 2024-04-26T20:04:44.022 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.022 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.134 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.134 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.134 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277018 s, 0.0 kB/s 2024-04-26T20:04:44.135 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.135 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.248 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155358 s, 0.0 kB/s 2024-04-26T20:04:44.249 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.249 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.297 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246536 s, 0.0 kB/s 2024-04-26T20:04:44.402 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.455 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.515 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226388 s, 0.0 kB/s 2024-04-26T20:04:44.516 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.517 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.624 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170357 s, 0.0 kB/s 2024-04-26T20:04:44.625 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.625 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020118 s, 0.0 kB/s 2024-04-26T20:04:44.782 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.783 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.834 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:44.897 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:44.897 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000366749 s, 0.0 kB/s 2024-04-26T20:04:44.899 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:44.899 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:44.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:44.950 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.006 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019342 s, 0.0 kB/s 2024-04-26T20:04:45.007 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.007 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.060 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.118 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.118 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162387 s, 0.0 kB/s 2024-04-26T20:04:45.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.119 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.168 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.272 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002356 s, 0.0 kB/s 2024-04-26T20:04:45.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.273 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.324 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.384 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.384 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.384 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257027 s, 0.0 kB/s 2024-04-26T20:04:45.385 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.440 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.501 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195618 s, 0.0 kB/s 2024-04-26T20:04:45.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.503 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.558 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.618 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152534 s, 0.0 kB/s 2024-04-26T20:04:45.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.620 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.737 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.738 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.738 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186469 s, 0.0 kB/s 2024-04-26T20:04:45.740 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.740 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.795 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.854 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180618 s, 0.0 kB/s 2024-04-26T20:04:45.855 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.855 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:45.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:45.907 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:45.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:45.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:45.968 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202279 s, 0.0 kB/s 2024-04-26T20:04:45.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:45.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.018 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.073 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159823 s, 0.0 kB/s 2024-04-26T20:04:46.074 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.074 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212764 s, 0.0 kB/s 2024-04-26T20:04:46.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.248 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191656 s, 0.0 kB/s 2024-04-26T20:04:46.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.365 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000313112 s, 0.0 kB/s 2024-04-26T20:04:46.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.476 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.537 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224421 s, 0.0 kB/s 2024-04-26T20:04:46.538 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.538 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000281816 s, 0.0 kB/s 2024-04-26T20:04:46.655 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.655 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.710 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175433 s, 0.0 kB/s 2024-04-26T20:04:46.774 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.774 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.823 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:46.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:46.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:46.928 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198789 s, 0.0 kB/s 2024-04-26T20:04:46.929 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:46.929 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:46.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:46.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.043 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015391 s, 0.0 kB/s 2024-04-26T20:04:47.044 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.044 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.095 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.156 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152475 s, 0.0 kB/s 2024-04-26T20:04:47.157 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.157 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.308 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.308 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175629 s, 0.0 kB/s 2024-04-26T20:04:47.309 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.309 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.363 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175625 s, 0.0 kB/s 2024-04-26T20:04:47.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.427 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.546 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237522 s, 0.0 kB/s 2024-04-26T20:04:47.547 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.547 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.704 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.705 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241057 s, 0.0 kB/s 2024-04-26T20:04:47.706 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.817 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.817 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185295 s, 0.0 kB/s 2024-04-26T20:04:47.818 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.818 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:47.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:47.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:47.929 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187783 s, 0.0 kB/s 2024-04-26T20:04:47.930 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:47.930 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:47.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:47.983 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.041 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.041 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156893 s, 0.0 kB/s 2024-04-26T20:04:48.042 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.042 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.093 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.155 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016307 s, 0.0 kB/s 2024-04-26T20:04:48.156 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.156 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.208 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.264 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179954 s, 0.0 kB/s 2024-04-26T20:04:48.265 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.265 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.378 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197285 s, 0.0 kB/s 2024-04-26T20:04:48.379 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.433 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197516 s, 0.0 kB/s 2024-04-26T20:04:48.495 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.495 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.629 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.695 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.696 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213604 s, 0.0 kB/s 2024-04-26T20:04:48.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.697 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.750 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155007 s, 0.0 kB/s 2024-04-26T20:04:48.808 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.808 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:48.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:48.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:48.956 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:48.956 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:48.957 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152548 s, 0.0 kB/s 2024-04-26T20:04:48.958 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:48.958 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.010 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.120 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020134 s, 0.0 kB/s 2024-04-26T20:04:49.121 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.233 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167468 s, 0.0 kB/s 2024-04-26T20:04:49.234 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.234 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.291 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196838 s, 0.0 kB/s 2024-04-26T20:04:49.350 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.350 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.464 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187967 s, 0.0 kB/s 2024-04-26T20:04:49.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.514 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.620 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222497 s, 0.0 kB/s 2024-04-26T20:04:49.621 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.621 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.673 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.730 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.730 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.730 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169275 s, 0.0 kB/s 2024-04-26T20:04:49.731 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.731 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216613 s, 0.0 kB/s 2024-04-26T20:04:49.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:49.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:49.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:49.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:49.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:49.966 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240121 s, 0.0 kB/s 2024-04-26T20:04:49.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:49.967 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.081 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178336 s, 0.0 kB/s 2024-04-26T20:04:50.082 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.230 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218675 s, 0.0 kB/s 2024-04-26T20:04:50.231 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.231 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.284 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209939 s, 0.0 kB/s 2024-04-26T20:04:50.345 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.396 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.462 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251856 s, 0.0 kB/s 2024-04-26T20:04:50.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.464 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.515 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.616 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194645 s, 0.0 kB/s 2024-04-26T20:04:50.617 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.617 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.666 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307026 s, 0.0 kB/s 2024-04-26T20:04:50.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.829 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:50.890 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:50.890 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:50.891 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167442 s, 0.0 kB/s 2024-04-26T20:04:50.892 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:50.892 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:50.943 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.001 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153675 s, 0.0 kB/s 2024-04-26T20:04:51.003 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.003 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.056 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241263 s, 0.0 kB/s 2024-04-26T20:04:51.121 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.170 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287802 s, 0.0 kB/s 2024-04-26T20:04:51.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.277 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.331 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.391 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030065 s, 0.0 kB/s 2024-04-26T20:04:51.392 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.392 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.444 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018771 s, 0.0 kB/s 2024-04-26T20:04:51.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.503 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.552 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.609 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022017 s, 0.0 kB/s 2024-04-26T20:04:51.610 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.610 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.664 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.723 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263149 s, 0.0 kB/s 2024-04-26T20:04:51.724 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.778 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.835 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148588 s, 0.0 kB/s 2024-04-26T20:04:51.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.836 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:51.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:51.885 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:51.989 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:51.989 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:51.989 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188768 s, 0.0 kB/s 2024-04-26T20:04:51.990 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:51.990 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.041 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.098 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162339 s, 0.0 kB/s 2024-04-26T20:04:52.099 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.099 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.147 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.251 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293264 s, 0.0 kB/s 2024-04-26T20:04:52.252 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.252 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.300 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.406 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.406 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250185 s, 0.0 kB/s 2024-04-26T20:04:52.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.458 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.568 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248088 s, 0.0 kB/s 2024-04-26T20:04:52.569 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.569 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.624 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235985 s, 0.0 kB/s 2024-04-26T20:04:52.687 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.687 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.788 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144214 s, 0.0 kB/s 2024-04-26T20:04:52.789 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.789 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:52.935 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205385 s, 0.0 kB/s 2024-04-26T20:04:52.936 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:52.936 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:52.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:52.988 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222759 s, 0.0 kB/s 2024-04-26T20:04:53.053 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.053 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.169 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.169 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.169 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022075 s, 0.0 kB/s 2024-04-26T20:04:53.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.170 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.227 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242839 s, 0.0 kB/s 2024-04-26T20:04:53.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.288 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.403 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208999 s, 0.0 kB/s 2024-04-26T20:04:53.404 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.404 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.458 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.522 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240546 s, 0.0 kB/s 2024-04-26T20:04:53.523 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.523 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.578 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.638 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214234 s, 0.0 kB/s 2024-04-26T20:04:53.639 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.639 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.746 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156882 s, 0.0 kB/s 2024-04-26T20:04:53.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.799 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:53.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:53.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:53.856 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000131103 s, 0.0 kB/s 2024-04-26T20:04:53.857 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:53.857 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:53.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:53.909 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212081 s, 0.0 kB/s 2024-04-26T20:04:54.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.014 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.127 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000325426 s, 0.0 kB/s 2024-04-26T20:04:54.128 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.182 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.240 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023025 s, 0.0 kB/s 2024-04-26T20:04:54.241 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.241 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.294 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217151 s, 0.0 kB/s 2024-04-26T20:04:54.352 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.352 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.403 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.465 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260238 s, 0.0 kB/s 2024-04-26T20:04:54.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.585 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000295737 s, 0.0 kB/s 2024-04-26T20:04:54.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.640 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019093 s, 0.0 kB/s 2024-04-26T20:04:54.699 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.699 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.750 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248009 s, 0.0 kB/s 2024-04-26T20:04:54.813 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.813 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.866 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:54.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:54.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:54.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181535 s, 0.0 kB/s 2024-04-26T20:04:54.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:54.928 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:54.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:54.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.038 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.038 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184918 s, 0.0 kB/s 2024-04-26T20:04:55.039 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.039 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.092 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.197 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.197 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.197 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179971 s, 0.0 kB/s 2024-04-26T20:04:55.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.198 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.244 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248075 s, 0.0 kB/s 2024-04-26T20:04:55.350 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.350 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.467 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258285 s, 0.0 kB/s 2024-04-26T20:04:55.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.519 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.622 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263297 s, 0.0 kB/s 2024-04-26T20:04:55.623 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.623 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.739 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184505 s, 0.0 kB/s 2024-04-26T20:04:55.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.741 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.793 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.850 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.850 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.851 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266368 s, 0.0 kB/s 2024-04-26T20:04:55.852 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:55.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:55.904 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:55.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:55.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:55.964 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239149 s, 0.0 kB/s 2024-04-26T20:04:55.965 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:55.965 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.019 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000135384 s, 0.0 kB/s 2024-04-26T20:04:56.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.079 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.188 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262098 s, 0.0 kB/s 2024-04-26T20:04:56.189 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.303 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204775 s, 0.0 kB/s 2024-04-26T20:04:56.304 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.304 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.360 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.420 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024322 s, 0.0 kB/s 2024-04-26T20:04:56.421 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.421 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.475 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.535 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280685 s, 0.0 kB/s 2024-04-26T20:04:56.536 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.537 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.589 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183167 s, 0.0 kB/s 2024-04-26T20:04:56.648 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.648 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.704 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.764 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179261 s, 0.0 kB/s 2024-04-26T20:04:56.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.766 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.819 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.878 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201346 s, 0.0 kB/s 2024-04-26T20:04:56.879 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.879 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:56.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:56.935 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:56.995 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175753 s, 0.0 kB/s 2024-04-26T20:04:56.996 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:56.996 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.048 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.109 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.109 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182633 s, 0.0 kB/s 2024-04-26T20:04:57.110 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.110 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.161 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208958 s, 0.0 kB/s 2024-04-26T20:04:57.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.379 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274826 s, 0.0 kB/s 2024-04-26T20:04:57.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.381 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.432 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.488 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.488 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.489 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237756 s, 0.0 kB/s 2024-04-26T20:04:57.490 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.491 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211183 s, 0.0 kB/s 2024-04-26T20:04:57.603 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.603 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.657 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.715 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.716 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186837 s, 0.0 kB/s 2024-04-26T20:04:57.717 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.717 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.769 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.831 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187889 s, 0.0 kB/s 2024-04-26T20:04:57.832 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.832 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.884 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:57.944 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:57.944 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:57.944 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253131 s, 0.0 kB/s 2024-04-26T20:04:57.945 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:57.945 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:57.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:57.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.058 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.058 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.058 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017765 s, 0.0 kB/s 2024-04-26T20:04:58.059 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.059 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.111 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.174 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198155 s, 0.0 kB/s 2024-04-26T20:04:58.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.176 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.230 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.288 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.288 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.288 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148525 s, 0.0 kB/s 2024-04-26T20:04:58.289 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.404 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155045 s, 0.0 kB/s 2024-04-26T20:04:58.405 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.406 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.457 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203786 s, 0.0 kB/s 2024-04-26T20:04:58.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.565 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.621 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192374 s, 0.0 kB/s 2024-04-26T20:04:58.622 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.622 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.669 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.777 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241371 s, 0.0 kB/s 2024-04-26T20:04:58.779 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.779 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:58.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:58.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:58.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218699 s, 0.0 kB/s 2024-04-26T20:04:58.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:58.890 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:58.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:58.941 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.001 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198597 s, 0.0 kB/s 2024-04-26T20:04:59.002 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.002 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.115 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231659 s, 0.0 kB/s 2024-04-26T20:04:59.117 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.117 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.230 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230427 s, 0.0 kB/s 2024-04-26T20:04:59.231 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.231 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.284 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.342 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194648 s, 0.0 kB/s 2024-04-26T20:04:59.344 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.344 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.395 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.455 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196749 s, 0.0 kB/s 2024-04-26T20:04:59.457 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.457 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.509 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223195 s, 0.0 kB/s 2024-04-26T20:04:59.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.686 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.687 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189469 s, 0.0 kB/s 2024-04-26T20:04:59.688 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.688 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.741 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230167 s, 0.0 kB/s 2024-04-26T20:04:59.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.803 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.855 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:04:59.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:04:59.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:04:59.914 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206546 s, 0.0 kB/s 2024-04-26T20:04:59.915 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:04:59.915 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:04:59.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:04:59.967 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174751 s, 0.0 kB/s 2024-04-26T20:05:00.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.121 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.228 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.228 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.228 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232658 s, 0.0 kB/s 2024-04-26T20:05:00.229 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.230 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.280 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306237 s, 0.0 kB/s 2024-04-26T20:05:00.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.439 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.500 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164001 s, 0.0 kB/s 2024-04-26T20:05:00.501 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.659 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244285 s, 0.0 kB/s 2024-04-26T20:05:00.660 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.660 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.713 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.776 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217616 s, 0.0 kB/s 2024-04-26T20:05:00.777 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.777 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:00.894 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214284 s, 0.0 kB/s 2024-04-26T20:05:00.896 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:00.896 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:00.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:00.949 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.007 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208705 s, 0.0 kB/s 2024-04-26T20:05:01.007 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.008 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.057 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.167 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.167 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219016 s, 0.0 kB/s 2024-04-26T20:05:01.168 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.168 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.222 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.282 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209056 s, 0.0 kB/s 2024-04-26T20:05:01.283 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.283 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.397 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166797 s, 0.0 kB/s 2024-04-26T20:05:01.398 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.398 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.508 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264406 s, 0.0 kB/s 2024-04-26T20:05:01.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.509 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.559 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239078 s, 0.0 kB/s 2024-04-26T20:05:01.619 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.619 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.668 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.771 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244567 s, 0.0 kB/s 2024-04-26T20:05:01.772 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.772 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.826 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245946 s, 0.0 kB/s 2024-04-26T20:05:01.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:01.890 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:01.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:01.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196343 s, 0.0 kB/s 2024-04-26T20:05:02.008 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.008 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.064 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.128 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021906 s, 0.0 kB/s 2024-04-26T20:05:02.129 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.181 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.239 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206299 s, 0.0 kB/s 2024-04-26T20:05:02.240 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.240 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.290 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.347 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178429 s, 0.0 kB/s 2024-04-26T20:05:02.348 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.458 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264308 s, 0.0 kB/s 2024-04-26T20:05:02.459 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.460 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.510 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155386 s, 0.0 kB/s 2024-04-26T20:05:02.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.625 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.679 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.680 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.680 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144647 s, 0.0 kB/s 2024-04-26T20:05:02.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.681 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278664 s, 0.0 kB/s 2024-04-26T20:05:02.798 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.799 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.851 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176839 s, 0.0 kB/s 2024-04-26T20:05:02.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:02.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:02.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:02.963 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.020 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233451 s, 0.0 kB/s 2024-04-26T20:05:03.021 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.022 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.075 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.139 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217575 s, 0.0 kB/s 2024-04-26T20:05:03.140 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.140 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.257 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000308735 s, 0.0 kB/s 2024-04-26T20:05:03.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.259 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.313 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257127 s, 0.0 kB/s 2024-04-26T20:05:03.374 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.425 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172889 s, 0.0 kB/s 2024-04-26T20:05:03.484 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.539 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193895 s, 0.0 kB/s 2024-04-26T20:05:03.598 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.598 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.645 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.747 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151206 s, 0.0 kB/s 2024-04-26T20:05:03.748 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.749 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.858 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185318 s, 0.0 kB/s 2024-04-26T20:05:03.858 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.858 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:03.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:03.912 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147852 s, 0.0 kB/s 2024-04-26T20:05:03.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:03.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184237 s, 0.0 kB/s 2024-04-26T20:05:04.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.204 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194928 s, 0.0 kB/s 2024-04-26T20:05:04.205 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.256 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.361 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000404467 s, 0.0 kB/s 2024-04-26T20:05:04.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.362 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.416 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.475 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177121 s, 0.0 kB/s 2024-04-26T20:05:04.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.531 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231527 s, 0.0 kB/s 2024-04-26T20:05:04.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015359 s, 0.0 kB/s 2024-04-26T20:05:04.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.742 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.856 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248077 s, 0.0 kB/s 2024-04-26T20:05:04.857 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.857 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:04.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:04.909 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:04.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:04.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:04.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212588 s, 0.0 kB/s 2024-04-26T20:05:04.971 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:04.971 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.087 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188169 s, 0.0 kB/s 2024-04-26T20:05:05.088 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.138 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238629 s, 0.0 kB/s 2024-04-26T20:05:05.244 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.244 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.299 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.359 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197622 s, 0.0 kB/s 2024-04-26T20:05:05.360 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.475 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190765 s, 0.0 kB/s 2024-04-26T20:05:05.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162115 s, 0.0 kB/s 2024-04-26T20:05:05.632 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.632 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.681 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166928 s, 0.0 kB/s 2024-04-26T20:05:05.789 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.789 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.841 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:05.898 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:05.898 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:05.898 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180581 s, 0.0 kB/s 2024-04-26T20:05:05.899 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:05.899 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:05.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:05.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.055 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197553 s, 0.0 kB/s 2024-04-26T20:05:06.056 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.056 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262147 s, 0.0 kB/s 2024-04-26T20:05:06.167 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.167 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.220 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.284 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193719 s, 0.0 kB/s 2024-04-26T20:05:06.285 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.285 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.336 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.392 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.392 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000298511 s, 0.0 kB/s 2024-04-26T20:05:06.393 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.393 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.446 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179363 s, 0.0 kB/s 2024-04-26T20:05:06.506 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.506 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.559 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.616 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151951 s, 0.0 kB/s 2024-04-26T20:05:06.617 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.618 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.671 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246498 s, 0.0 kB/s 2024-04-26T20:05:06.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:06.848 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191939 s, 0.0 kB/s 2024-04-26T20:05:06.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:06.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:06.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:06.901 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.008 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179415 s, 0.0 kB/s 2024-04-26T20:05:07.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.009 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.065 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.126 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146956 s, 0.0 kB/s 2024-04-26T20:05:07.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.127 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.282 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207685 s, 0.0 kB/s 2024-04-26T20:05:07.283 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.283 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160664 s, 0.0 kB/s 2024-04-26T20:05:07.402 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.457 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.517 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187155 s, 0.0 kB/s 2024-04-26T20:05:07.518 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.518 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.570 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.629 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194897 s, 0.0 kB/s 2024-04-26T20:05:07.630 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.630 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000126772 s, 0.0 kB/s 2024-04-26T20:05:07.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.744 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.861 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203317 s, 0.0 kB/s 2024-04-26T20:05:07.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.863 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:07.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:07.917 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:07.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:07.980 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025997 s, 0.0 kB/s 2024-04-26T20:05:07.981 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:07.982 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.037 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178751 s, 0.0 kB/s 2024-04-26T20:05:08.097 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.098 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.147 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209896 s, 0.0 kB/s 2024-04-26T20:05:08.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.257 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164054 s, 0.0 kB/s 2024-04-26T20:05:08.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.359 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.475 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022103 s, 0.0 kB/s 2024-04-26T20:05:08.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.531 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217266 s, 0.0 kB/s 2024-04-26T20:05:08.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.646 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145957 s, 0.0 kB/s 2024-04-26T20:05:08.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192562 s, 0.0 kB/s 2024-04-26T20:05:08.819 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.819 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:08.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:08.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000281672 s, 0.0 kB/s 2024-04-26T20:05:08.977 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:08.977 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.032 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.093 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183928 s, 0.0 kB/s 2024-04-26T20:05:09.093 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.094 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.145 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.205 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166114 s, 0.0 kB/s 2024-04-26T20:05:09.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.206 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.358 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201617 s, 0.0 kB/s 2024-04-26T20:05:09.359 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.415 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.476 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.477 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.477 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246405 s, 0.0 kB/s 2024-04-26T20:05:09.478 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.478 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.592 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186475 s, 0.0 kB/s 2024-04-26T20:05:09.593 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.593 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.642 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.745 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.746 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016269 s, 0.0 kB/s 2024-04-26T20:05:09.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.795 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:09.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:09.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:09.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229447 s, 0.0 kB/s 2024-04-26T20:05:09.902 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:09.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:09.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:09.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189182 s, 0.0 kB/s 2024-04-26T20:05:10.010 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.010 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.061 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.120 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169689 s, 0.0 kB/s 2024-04-26T20:05:10.121 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.235 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.235 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258004 s, 0.0 kB/s 2024-04-26T20:05:10.236 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.236 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.291 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.352 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257364 s, 0.0 kB/s 2024-04-26T20:05:10.353 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.353 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.407 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188779 s, 0.0 kB/s 2024-04-26T20:05:10.475 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.475 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.526 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217291 s, 0.0 kB/s 2024-04-26T20:05:10.587 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.587 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.702 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254811 s, 0.0 kB/s 2024-04-26T20:05:10.703 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.703 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.753 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:10.760 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:05:10.807 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:05:10.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:10.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:10.815 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00013854 s, 0.0 kB/s 2024-04-26T20:05:10.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:10.816 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:10.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:10.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.013 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.014 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.014 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204826 s, 0.0 kB/s 2024-04-26T20:05:11.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.125 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.126 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149753 s, 0.0 kB/s 2024-04-26T20:05:11.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.127 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.177 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.236 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.236 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.236 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201159 s, 0.0 kB/s 2024-04-26T20:05:11.237 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.237 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.290 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150978 s, 0.0 kB/s 2024-04-26T20:05:11.350 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.350 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.405 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.475 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242974 s, 0.0 kB/s 2024-04-26T20:05:11.477 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.477 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.592 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171772 s, 0.0 kB/s 2024-04-26T20:05:11.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.707 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164329 s, 0.0 kB/s 2024-04-26T20:05:11.708 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.708 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.760 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.820 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238532 s, 0.0 kB/s 2024-04-26T20:05:11.821 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.821 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.873 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:11.931 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:11.932 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:11.932 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162421 s, 0.0 kB/s 2024-04-26T20:05:11.933 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:11.933 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:11.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:11.985 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.048 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191148 s, 0.0 kB/s 2024-04-26T20:05:12.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.049 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.101 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178857 s, 0.0 kB/s 2024-04-26T20:05:12.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.206 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.256 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.362 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.362 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193978 s, 0.0 kB/s 2024-04-26T20:05:12.363 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.363 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.416 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220411 s, 0.0 kB/s 2024-04-26T20:05:12.479 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.479 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198071 s, 0.0 kB/s 2024-04-26T20:05:12.634 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.634 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.692 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.748 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259323 s, 0.0 kB/s 2024-04-26T20:05:12.749 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.749 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.799 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:12.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:12.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:12.903 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219758 s, 0.0 kB/s 2024-04-26T20:05:12.904 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:12.904 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:12.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:12.957 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.014 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.014 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.014 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214128 s, 0.0 kB/s 2024-04-26T20:05:13.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.065 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.123 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214463 s, 0.0 kB/s 2024-04-26T20:05:13.124 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.124 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.177 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.240 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184741 s, 0.0 kB/s 2024-04-26T20:05:13.241 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.242 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185536 s, 0.0 kB/s 2024-04-26T20:05:13.358 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.358 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.479 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.480 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209677 s, 0.0 kB/s 2024-04-26T20:05:13.482 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.482 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.536 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.598 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156657 s, 0.0 kB/s 2024-04-26T20:05:13.599 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.599 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.651 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.709 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000327033 s, 0.0 kB/s 2024-04-26T20:05:13.711 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.764 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.821 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154375 s, 0.0 kB/s 2024-04-26T20:05:13.822 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:13.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:13.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:13.976 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304073 s, 0.0 kB/s 2024-04-26T20:05:13.977 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:13.977 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.034 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.094 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176931 s, 0.0 kB/s 2024-04-26T20:05:14.095 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.095 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.151 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.212 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221883 s, 0.0 kB/s 2024-04-26T20:05:14.213 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.213 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.267 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.326 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155877 s, 0.0 kB/s 2024-04-26T20:05:14.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.380 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227654 s, 0.0 kB/s 2024-04-26T20:05:14.443 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.444 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.494 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.598 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186873 s, 0.0 kB/s 2024-04-26T20:05:14.600 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.600 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.655 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.715 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.716 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217221 s, 0.0 kB/s 2024-04-26T20:05:14.717 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.717 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.768 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.824 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178385 s, 0.0 kB/s 2024-04-26T20:05:14.825 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.825 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.877 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:14.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:14.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:14.935 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146789 s, 0.0 kB/s 2024-04-26T20:05:14.936 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:14.936 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:14.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:14.986 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.042 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185453 s, 0.0 kB/s 2024-04-26T20:05:15.043 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.044 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.153 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162274 s, 0.0 kB/s 2024-04-26T20:05:15.154 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.154 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.207 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.262 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156652 s, 0.0 kB/s 2024-04-26T20:05:15.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.309 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165668 s, 0.0 kB/s 2024-04-26T20:05:15.413 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.413 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.466 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.530 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212247 s, 0.0 kB/s 2024-04-26T20:05:15.531 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.531 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.580 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.681 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244407 s, 0.0 kB/s 2024-04-26T20:05:15.682 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.682 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.728 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194933 s, 0.0 kB/s 2024-04-26T20:05:15.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.888 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:15.946 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:15.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:15.947 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235102 s, 0.0 kB/s 2024-04-26T20:05:15.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:15.948 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:15.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:15.998 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.056 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264909 s, 0.0 kB/s 2024-04-26T20:05:16.058 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.058 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.111 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.173 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176094 s, 0.0 kB/s 2024-04-26T20:05:16.174 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.174 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.225 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.329 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219425 s, 0.0 kB/s 2024-04-26T20:05:16.330 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.330 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.445 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161249 s, 0.0 kB/s 2024-04-26T20:05:16.446 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.597 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.597 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183217 s, 0.0 kB/s 2024-04-26T20:05:16.598 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.598 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.650 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.709 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000143579 s, 0.0 kB/s 2024-04-26T20:05:16.709 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.709 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.762 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179967 s, 0.0 kB/s 2024-04-26T20:05:16.822 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.875 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:16.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:16.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:16.934 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144279 s, 0.0 kB/s 2024-04-26T20:05:16.935 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:16.935 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:16.985 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159377 s, 0.0 kB/s 2024-04-26T20:05:17.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.090 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.203 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194235 s, 0.0 kB/s 2024-04-26T20:05:17.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.204 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233815 s, 0.0 kB/s 2024-04-26T20:05:17.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.363 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.468 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240527 s, 0.0 kB/s 2024-04-26T20:05:17.469 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.469 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.522 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.582 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184682 s, 0.0 kB/s 2024-04-26T20:05:17.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182574 s, 0.0 kB/s 2024-04-26T20:05:17.701 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.702 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.754 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.819 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190601 s, 0.0 kB/s 2024-04-26T20:05:17.820 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.820 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.877 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:17.941 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:17.941 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:17.941 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176451 s, 0.0 kB/s 2024-04-26T20:05:17.942 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:17.942 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:17.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:17.995 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.051 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219466 s, 0.0 kB/s 2024-04-26T20:05:18.052 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.052 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199864 s, 0.0 kB/s 2024-04-26T20:05:18.167 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.167 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.218 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186533 s, 0.0 kB/s 2024-04-26T20:05:18.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.277 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.333 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.395 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213632 s, 0.0 kB/s 2024-04-26T20:05:18.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.448 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.503 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.503 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181313 s, 0.0 kB/s 2024-04-26T20:05:18.504 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.504 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.551 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.656 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166597 s, 0.0 kB/s 2024-04-26T20:05:18.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.657 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.709 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.770 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241074 s, 0.0 kB/s 2024-04-26T20:05:18.774 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.827 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:18.886 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:18.886 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:18.886 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181558 s, 0.0 kB/s 2024-04-26T20:05:18.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:18.887 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:18.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:18.941 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.003 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237297 s, 0.0 kB/s 2024-04-26T20:05:19.005 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.005 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.059 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.120 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261863 s, 0.0 kB/s 2024-04-26T20:05:19.121 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199064 s, 0.0 kB/s 2024-04-26T20:05:19.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.352 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.421 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208334 s, 0.0 kB/s 2024-04-26T20:05:19.422 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.422 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.474 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000135775 s, 0.0 kB/s 2024-04-26T20:05:19.579 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.579 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.631 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022787 s, 0.0 kB/s 2024-04-26T20:05:19.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.785 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.845 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291449 s, 0.0 kB/s 2024-04-26T20:05:19.847 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.847 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:19.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:19.899 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232026 s, 0.0 kB/s 2024-04-26T20:05:19.963 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:19.964 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280951 s, 0.0 kB/s 2024-04-26T20:05:20.084 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.084 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.138 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.201 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.202 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208493 s, 0.0 kB/s 2024-04-26T20:05:20.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.203 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.312 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184716 s, 0.0 kB/s 2024-04-26T20:05:20.313 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.313 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024302 s, 0.0 kB/s 2024-04-26T20:05:20.432 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.432 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.485 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.547 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172985 s, 0.0 kB/s 2024-04-26T20:05:20.548 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.548 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.660 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.660 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235681 s, 0.0 kB/s 2024-04-26T20:05:20.661 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.661 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.714 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.775 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250541 s, 0.0 kB/s 2024-04-26T20:05:20.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.776 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.825 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:20.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:20.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:20.884 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222186 s, 0.0 kB/s 2024-04-26T20:05:20.886 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:20.886 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:20.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:20.939 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219891 s, 0.0 kB/s 2024-04-26T20:05:21.003 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.003 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230182 s, 0.0 kB/s 2024-04-26T20:05:21.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.161 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.269 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215063 s, 0.0 kB/s 2024-04-26T20:05:21.270 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.270 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.323 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.381 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.381 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000350748 s, 0.0 kB/s 2024-04-26T20:05:21.382 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.382 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.433 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240169 s, 0.0 kB/s 2024-04-26T20:05:21.495 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.495 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.547 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.608 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024765 s, 0.0 kB/s 2024-04-26T20:05:21.609 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.609 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.666 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.723 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209623 s, 0.0 kB/s 2024-04-26T20:05:21.724 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.775 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.838 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213004 s, 0.0 kB/s 2024-04-26T20:05:21.839 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.839 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:21.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:21.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:21.988 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021239 s, 0.0 kB/s 2024-04-26T20:05:21.989 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:21.990 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.041 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.145 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161162 s, 0.0 kB/s 2024-04-26T20:05:22.146 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.146 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261651 s, 0.0 kB/s 2024-04-26T20:05:22.303 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.303 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.356 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.417 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018353 s, 0.0 kB/s 2024-04-26T20:05:22.418 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.418 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.525 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152967 s, 0.0 kB/s 2024-04-26T20:05:22.526 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.526 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224132 s, 0.0 kB/s 2024-04-26T20:05:22.683 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.683 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.739 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.798 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178821 s, 0.0 kB/s 2024-04-26T20:05:22.799 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.799 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.852 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168512 s, 0.0 kB/s 2024-04-26T20:05:22.912 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:22.913 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:22.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:22.963 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.023 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.023 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000135594 s, 0.0 kB/s 2024-04-26T20:05:23.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.024 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.081 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.142 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150159 s, 0.0 kB/s 2024-04-26T20:05:23.143 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.143 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.254 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.255 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.255 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258963 s, 0.0 kB/s 2024-04-26T20:05:23.256 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.256 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.310 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186249 s, 0.0 kB/s 2024-04-26T20:05:23.373 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.373 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.425 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.486 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.486 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018301 s, 0.0 kB/s 2024-04-26T20:05:23.487 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.487 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.539 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.594 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263235 s, 0.0 kB/s 2024-04-26T20:05:23.595 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199487 s, 0.0 kB/s 2024-04-26T20:05:23.710 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.710 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.762 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.821 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171544 s, 0.0 kB/s 2024-04-26T20:05:23.822 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:23.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:23.867 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022927 s, 0.0 kB/s 2024-04-26T20:05:23.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:23.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174254 s, 0.0 kB/s 2024-04-26T20:05:24.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.135 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150943 s, 0.0 kB/s 2024-04-26T20:05:24.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.250 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.310 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150163 s, 0.0 kB/s 2024-04-26T20:05:24.311 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.464 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021992 s, 0.0 kB/s 2024-04-26T20:05:24.465 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.584 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284396 s, 0.0 kB/s 2024-04-26T20:05:24.585 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.585 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.692 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221598 s, 0.0 kB/s 2024-04-26T20:05:24.693 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.693 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.745 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.805 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228351 s, 0.0 kB/s 2024-04-26T20:05:24.807 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.807 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:24.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:24.857 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:24.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:24.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:24.958 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212731 s, 0.0 kB/s 2024-04-26T20:05:24.959 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:24.959 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.009 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.114 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.115 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00028011 s, 0.0 kB/s 2024-04-26T20:05:25.115 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.115 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.170 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165502 s, 0.0 kB/s 2024-04-26T20:05:25.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.277 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.382 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000142237 s, 0.0 kB/s 2024-04-26T20:05:25.383 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.432 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.536 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160889 s, 0.0 kB/s 2024-04-26T20:05:25.537 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.537 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.590 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.647 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181445 s, 0.0 kB/s 2024-04-26T20:05:25.648 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.648 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.698 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235809 s, 0.0 kB/s 2024-04-26T20:05:25.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.819 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:25.881 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206244 s, 0.0 kB/s 2024-04-26T20:05:25.882 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:25.883 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:25.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:25.942 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.003 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190264 s, 0.0 kB/s 2024-04-26T20:05:26.004 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.004 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.056 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.116 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235074 s, 0.0 kB/s 2024-04-26T20:05:26.117 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.117 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.169 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189635 s, 0.0 kB/s 2024-04-26T20:05:26.276 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.276 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.332 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.395 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183171 s, 0.0 kB/s 2024-04-26T20:05:26.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150125 s, 0.0 kB/s 2024-04-26T20:05:26.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.571 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.628 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193719 s, 0.0 kB/s 2024-04-26T20:05:26.629 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.629 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.684 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195637 s, 0.0 kB/s 2024-04-26T20:05:26.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.744 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187036 s, 0.0 kB/s 2024-04-26T20:05:26.856 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:26.856 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:26.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:26.908 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168921 s, 0.0 kB/s 2024-04-26T20:05:27.014 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.014 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.071 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274007 s, 0.0 kB/s 2024-04-26T20:05:27.136 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.136 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.188 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216921 s, 0.0 kB/s 2024-04-26T20:05:27.246 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.246 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.400 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207419 s, 0.0 kB/s 2024-04-26T20:05:27.401 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.401 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.517 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215468 s, 0.0 kB/s 2024-04-26T20:05:27.518 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.518 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.570 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201731 s, 0.0 kB/s 2024-04-26T20:05:27.631 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.631 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.742 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260359 s, 0.0 kB/s 2024-04-26T20:05:27.743 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.743 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190892 s, 0.0 kB/s 2024-04-26T20:05:27.860 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:27.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:27.910 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:27.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:27.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:27.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224386 s, 0.0 kB/s 2024-04-26T20:05:27.968 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:27.968 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246938 s, 0.0 kB/s 2024-04-26T20:05:28.088 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.088 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.143 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212385 s, 0.0 kB/s 2024-04-26T20:05:28.202 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.250 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.354 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245675 s, 0.0 kB/s 2024-04-26T20:05:28.355 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.355 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.512 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151329 s, 0.0 kB/s 2024-04-26T20:05:28.513 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.565 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232067 s, 0.0 kB/s 2024-04-26T20:05:28.626 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.627 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.738 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000305875 s, 0.0 kB/s 2024-04-26T20:05:28.739 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.739 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.793 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174574 s, 0.0 kB/s 2024-04-26T20:05:28.854 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.855 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:28.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:28.909 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:28.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:28.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:28.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022357 s, 0.0 kB/s 2024-04-26T20:05:28.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:28.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000139434 s, 0.0 kB/s 2024-04-26T20:05:29.081 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.131 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.189 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.189 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165429 s, 0.0 kB/s 2024-04-26T20:05:29.190 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.190 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.240 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199689 s, 0.0 kB/s 2024-04-26T20:05:29.345 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.396 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254127 s, 0.0 kB/s 2024-04-26T20:05:29.462 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.462 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.515 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.573 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.573 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.573 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172138 s, 0.0 kB/s 2024-04-26T20:05:29.574 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.574 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.629 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158017 s, 0.0 kB/s 2024-04-26T20:05:29.689 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.689 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.741 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184744 s, 0.0 kB/s 2024-04-26T20:05:29.801 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.801 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:29.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:29.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:29.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189125 s, 0.0 kB/s 2024-04-26T20:05:29.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:29.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:29.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:29.964 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167656 s, 0.0 kB/s 2024-04-26T20:05:30.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.124 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218219 s, 0.0 kB/s 2024-04-26T20:05:30.188 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.188 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.306 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241242 s, 0.0 kB/s 2024-04-26T20:05:30.308 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.364 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.423 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.423 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234793 s, 0.0 kB/s 2024-04-26T20:05:30.424 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.424 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.542 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.542 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.543 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180801 s, 0.0 kB/s 2024-04-26T20:05:30.543 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.544 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.595 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.651 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.652 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.652 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199902 s, 0.0 kB/s 2024-04-26T20:05:30.653 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.653 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.705 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.766 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226917 s, 0.0 kB/s 2024-04-26T20:05:30.767 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.767 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.821 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.880 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.880 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.880 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220449 s, 0.0 kB/s 2024-04-26T20:05:30.882 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.882 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:30.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:30.935 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:30.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:30.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:30.994 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192292 s, 0.0 kB/s 2024-04-26T20:05:30.995 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:30.995 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.046 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198926 s, 0.0 kB/s 2024-04-26T20:05:31.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.113 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.233 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197061 s, 0.0 kB/s 2024-04-26T20:05:31.234 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.234 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.286 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184851 s, 0.0 kB/s 2024-04-26T20:05:31.348 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.398 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.500 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.501 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197747 s, 0.0 kB/s 2024-04-26T20:05:31.502 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.502 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.618 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242739 s, 0.0 kB/s 2024-04-26T20:05:31.619 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.619 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.672 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.731 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.731 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.731 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255521 s, 0.0 kB/s 2024-04-26T20:05:31.732 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.732 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.848 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202878 s, 0.0 kB/s 2024-04-26T20:05:31.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:31.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:31.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:31.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:31.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000298844 s, 0.0 kB/s 2024-04-26T20:05:31.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:31.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.086 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190017 s, 0.0 kB/s 2024-04-26T20:05:32.088 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.088 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.205 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170626 s, 0.0 kB/s 2024-04-26T20:05:32.207 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.207 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.262 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238335 s, 0.0 kB/s 2024-04-26T20:05:32.318 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.318 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.368 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.472 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196762 s, 0.0 kB/s 2024-04-26T20:05:32.474 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.474 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.527 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229807 s, 0.0 kB/s 2024-04-26T20:05:32.587 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.588 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.701 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163137 s, 0.0 kB/s 2024-04-26T20:05:32.702 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.702 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173354 s, 0.0 kB/s 2024-04-26T20:05:32.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:32.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:32.908 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:32.968 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167881 s, 0.0 kB/s 2024-04-26T20:05:32.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:32.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020695 s, 0.0 kB/s 2024-04-26T20:05:33.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.079 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.133 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.192 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.192 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.192 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247742 s, 0.0 kB/s 2024-04-26T20:05:33.193 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.194 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.242 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263311 s, 0.0 kB/s 2024-04-26T20:05:33.353 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.353 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.467 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182918 s, 0.0 kB/s 2024-04-26T20:05:33.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025145 s, 0.0 kB/s 2024-04-26T20:05:33.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.694 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261062 s, 0.0 kB/s 2024-04-26T20:05:33.695 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.695 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.743 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.849 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.849 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.849 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000297099 s, 0.0 kB/s 2024-04-26T20:05:33.851 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.851 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:33.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:33.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:33.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:33.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:33.960 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277792 s, 0.0 kB/s 2024-04-26T20:05:33.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:33.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149566 s, 0.0 kB/s 2024-04-26T20:05:34.075 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.184 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196865 s, 0.0 kB/s 2024-04-26T20:05:34.185 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.240 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.300 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000118545 s, 0.0 kB/s 2024-04-26T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.301 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.353 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.410 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180723 s, 0.0 kB/s 2024-04-26T20:05:34.411 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.411 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.463 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.526 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.526 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001671 s, 0.0 kB/s 2024-04-26T20:05:34.527 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.527 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.582 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.643 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015981 s, 0.0 kB/s 2024-04-26T20:05:34.644 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.644 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.701 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000128337 s, 0.0 kB/s 2024-04-26T20:05:34.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.764 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.817 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.879 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268219 s, 0.0 kB/s 2024-04-26T20:05:34.880 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.880 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:34.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:34.934 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:34.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:34.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:34.995 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200476 s, 0.0 kB/s 2024-04-26T20:05:34.996 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:34.996 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.052 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.112 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217877 s, 0.0 kB/s 2024-04-26T20:05:35.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.114 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.166 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.221 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227639 s, 0.0 kB/s 2024-04-26T20:05:35.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.223 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.274 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217131 s, 0.0 kB/s 2024-04-26T20:05:35.334 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.335 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.442 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164693 s, 0.0 kB/s 2024-04-26T20:05:35.443 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.443 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.491 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220081 s, 0.0 kB/s 2024-04-26T20:05:35.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.706 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216045 s, 0.0 kB/s 2024-04-26T20:05:35.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.758 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.821 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215996 s, 0.0 kB/s 2024-04-26T20:05:35.822 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.823 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:35.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:35.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:35.976 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:35.976 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:35.976 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203341 s, 0.0 kB/s 2024-04-26T20:05:35.978 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:35.978 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.028 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.086 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018863 s, 0.0 kB/s 2024-04-26T20:05:36.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.087 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.138 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023413 s, 0.0 kB/s 2024-04-26T20:05:36.196 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.353 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00049939 s, 0.0 kB/s 2024-04-26T20:05:36.354 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.355 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.409 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.474 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312461 s, 0.0 kB/s 2024-04-26T20:05:36.475 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.475 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.529 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.589 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264529 s, 0.0 kB/s 2024-04-26T20:05:36.590 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.590 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.643 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.708 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024575 s, 0.0 kB/s 2024-04-26T20:05:36.709 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.709 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.761 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.815 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148136 s, 0.0 kB/s 2024-04-26T20:05:36.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.816 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:36.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:36.866 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:36.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018514 s, 0.0 kB/s 2024-04-26T20:05:36.970 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:36.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.023 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.081 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000140134 s, 0.0 kB/s 2024-04-26T20:05:37.082 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.130 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164964 s, 0.0 kB/s 2024-04-26T20:05:37.237 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.237 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.289 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.348 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018839 s, 0.0 kB/s 2024-04-26T20:05:37.350 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.350 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.403 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.465 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239403 s, 0.0 kB/s 2024-04-26T20:05:37.466 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.466 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211932 s, 0.0 kB/s 2024-04-26T20:05:37.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171873 s, 0.0 kB/s 2024-04-26T20:05:37.703 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.703 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.755 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.815 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144312 s, 0.0 kB/s 2024-04-26T20:05:37.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.816 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.866 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218823 s, 0.0 kB/s 2024-04-26T20:05:37.926 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:37.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:37.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:37.972 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.079 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278009 s, 0.0 kB/s 2024-04-26T20:05:38.080 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.136 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.195 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175831 s, 0.0 kB/s 2024-04-26T20:05:38.197 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.197 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.248 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231264 s, 0.0 kB/s 2024-04-26T20:05:38.354 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.355 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.408 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021297 s, 0.0 kB/s 2024-04-26T20:05:38.467 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.467 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.582 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243412 s, 0.0 kB/s 2024-04-26T20:05:38.583 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.583 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.634 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.697 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219694 s, 0.0 kB/s 2024-04-26T20:05:38.698 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.698 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219988 s, 0.0 kB/s 2024-04-26T20:05:38.809 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.809 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:38.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:38.858 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:38.963 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:38.963 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:38.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018469 s, 0.0 kB/s 2024-04-26T20:05:38.964 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:38.964 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.071 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184461 s, 0.0 kB/s 2024-04-26T20:05:39.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.072 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.184 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166587 s, 0.0 kB/s 2024-04-26T20:05:39.185 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.240 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.294 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196135 s, 0.0 kB/s 2024-04-26T20:05:39.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.296 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.348 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.406 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.406 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.406 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219664 s, 0.0 kB/s 2024-04-26T20:05:39.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.458 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.565 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.565 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.566 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270455 s, 0.0 kB/s 2024-04-26T20:05:39.567 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.567 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.624 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.684 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280209 s, 0.0 kB/s 2024-04-26T20:05:39.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.743 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.805 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166828 s, 0.0 kB/s 2024-04-26T20:05:39.806 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.806 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:39.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:39.857 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:39.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:39.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:39.962 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211077 s, 0.0 kB/s 2024-04-26T20:05:39.963 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:39.963 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156532 s, 0.0 kB/s 2024-04-26T20:05:40.075 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.124 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.231 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175559 s, 0.0 kB/s 2024-04-26T20:05:40.233 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.234 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.283 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.386 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194738 s, 0.0 kB/s 2024-04-26T20:05:40.387 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.387 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.439 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178921 s, 0.0 kB/s 2024-04-26T20:05:40.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.503 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.557 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.619 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.619 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206209 s, 0.0 kB/s 2024-04-26T20:05:40.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.620 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.674 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.736 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017669 s, 0.0 kB/s 2024-04-26T20:05:40.737 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.737 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186307 s, 0.0 kB/s 2024-04-26T20:05:40.850 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:40.850 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:40.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:40.901 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:40.948 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:05:40.993 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:05:41.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.003 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197232 s, 0.0 kB/s 2024-04-26T20:05:41.005 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.005 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.095 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306264 s, 0.0 kB/s 2024-04-26T20:05:41.201 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.201 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.257 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254818 s, 0.0 kB/s 2024-04-26T20:05:41.316 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.316 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.367 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.431 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.431 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018987 s, 0.0 kB/s 2024-04-26T20:05:41.432 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.432 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216524 s, 0.0 kB/s 2024-04-26T20:05:41.539 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.539 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.592 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.654 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182554 s, 0.0 kB/s 2024-04-26T20:05:41.655 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.655 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.710 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164477 s, 0.0 kB/s 2024-04-26T20:05:41.774 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.774 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.830 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181347 s, 0.0 kB/s 2024-04-26T20:05:41.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:41.890 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:41.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:41.942 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.002 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.002 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219648 s, 0.0 kB/s 2024-04-26T20:05:42.003 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.003 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.050 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188057 s, 0.0 kB/s 2024-04-26T20:05:42.155 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.155 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180587 s, 0.0 kB/s 2024-04-26T20:05:42.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.366 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235101 s, 0.0 kB/s 2024-04-26T20:05:42.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.476 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190145 s, 0.0 kB/s 2024-04-26T20:05:42.539 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.539 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.649 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192418 s, 0.0 kB/s 2024-04-26T20:05:42.650 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.650 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.703 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.809 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.809 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172389 s, 0.0 kB/s 2024-04-26T20:05:42.810 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.810 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.865 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:42.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:42.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:42.928 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173037 s, 0.0 kB/s 2024-04-26T20:05:42.929 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:42.929 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:42.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:42.983 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.046 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153989 s, 0.0 kB/s 2024-04-26T20:05:43.048 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.048 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.103 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186757 s, 0.0 kB/s 2024-04-26T20:05:43.166 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.166 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.218 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.280 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191765 s, 0.0 kB/s 2024-04-26T20:05:43.281 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.281 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.338 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.402 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.402 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.403 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192572 s, 0.0 kB/s 2024-04-26T20:05:43.403 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.404 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.519 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.519 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.519 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260871 s, 0.0 kB/s 2024-04-26T20:05:43.520 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.521 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.569 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.669 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.669 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.669 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169031 s, 0.0 kB/s 2024-04-26T20:05:43.670 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.671 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.722 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.826 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.826 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.826 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160705 s, 0.0 kB/s 2024-04-26T20:05:43.827 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.878 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:43.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:43.934 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:43.934 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221734 s, 0.0 kB/s 2024-04-26T20:05:43.935 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:43.936 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:43.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:43.988 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.044 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191247 s, 0.0 kB/s 2024-04-26T20:05:44.045 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.046 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.099 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245651 s, 0.0 kB/s 2024-04-26T20:05:44.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.160 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.214 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.273 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218915 s, 0.0 kB/s 2024-04-26T20:05:44.274 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.275 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.324 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.427 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.428 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.428 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198162 s, 0.0 kB/s 2024-04-26T20:05:44.429 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.429 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.486 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.547 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256767 s, 0.0 kB/s 2024-04-26T20:05:44.548 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.548 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.664 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00036128 s, 0.0 kB/s 2024-04-26T20:05:44.665 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.666 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227187 s, 0.0 kB/s 2024-04-26T20:05:44.780 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.831 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:44.938 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:44.938 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:44.938 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244117 s, 0.0 kB/s 2024-04-26T20:05:44.939 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:44.939 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:44.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:44.991 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.048 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000140076 s, 0.0 kB/s 2024-04-26T20:05:45.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.049 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.100 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015417 s, 0.0 kB/s 2024-04-26T20:05:45.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.160 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.273 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167087 s, 0.0 kB/s 2024-04-26T20:05:45.274 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.274 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.388 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229332 s, 0.0 kB/s 2024-04-26T20:05:45.389 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.389 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.440 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185041 s, 0.0 kB/s 2024-04-26T20:05:45.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.503 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.615 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195591 s, 0.0 kB/s 2024-04-26T20:05:45.616 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.617 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.667 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.726 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153023 s, 0.0 kB/s 2024-04-26T20:05:45.728 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.728 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.779 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170939 s, 0.0 kB/s 2024-04-26T20:05:45.842 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.842 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:45.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:45.892 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:45.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:45.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:45.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199362 s, 0.0 kB/s 2024-04-26T20:05:45.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:45.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.006 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.071 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000346022 s, 0.0 kB/s 2024-04-26T20:05:46.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.072 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.185 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.185 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184405 s, 0.0 kB/s 2024-04-26T20:05:46.186 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.186 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181102 s, 0.0 kB/s 2024-04-26T20:05:46.302 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.302 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.354 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.460 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237084 s, 0.0 kB/s 2024-04-26T20:05:46.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.461 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.519 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222659 s, 0.0 kB/s 2024-04-26T20:05:46.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228043 s, 0.0 kB/s 2024-04-26T20:05:46.698 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.698 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.815 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021302 s, 0.0 kB/s 2024-04-26T20:05:46.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.816 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:46.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:46.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:46.923 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000131735 s, 0.0 kB/s 2024-04-26T20:05:46.924 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:46.924 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:46.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:46.972 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247137 s, 0.0 kB/s 2024-04-26T20:05:47.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.079 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.135 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156119 s, 0.0 kB/s 2024-04-26T20:05:47.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.193 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.249 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.305 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144549 s, 0.0 kB/s 2024-04-26T20:05:47.306 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.306 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.353 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.458 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000336692 s, 0.0 kB/s 2024-04-26T20:05:47.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.460 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.511 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265072 s, 0.0 kB/s 2024-04-26T20:05:47.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.626 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149543 s, 0.0 kB/s 2024-04-26T20:05:47.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.733 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021212 s, 0.0 kB/s 2024-04-26T20:05:47.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.836 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:47.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:47.891 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:47.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:47.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:47.949 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217711 s, 0.0 kB/s 2024-04-26T20:05:47.950 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:47.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.106 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260481 s, 0.0 kB/s 2024-04-26T20:05:48.107 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.107 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.160 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000351279 s, 0.0 kB/s 2024-04-26T20:05:48.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.223 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.274 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.330 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.330 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.330 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177519 s, 0.0 kB/s 2024-04-26T20:05:48.331 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.331 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183381 s, 0.0 kB/s 2024-04-26T20:05:48.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.500 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.559 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000276036 s, 0.0 kB/s 2024-04-26T20:05:48.560 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.560 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.676 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017725 s, 0.0 kB/s 2024-04-26T20:05:48.677 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.677 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.731 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.794 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200041 s, 0.0 kB/s 2024-04-26T20:05:48.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.795 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:48.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:48.844 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:48.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:48.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:48.950 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186102 s, 0.0 kB/s 2024-04-26T20:05:48.951 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:48.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.004 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238699 s, 0.0 kB/s 2024-04-26T20:05:49.070 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.070 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.123 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.182 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018235 s, 0.0 kB/s 2024-04-26T20:05:49.184 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.184 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.236 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228403 s, 0.0 kB/s 2024-04-26T20:05:49.302 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.302 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.354 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.414 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197602 s, 0.0 kB/s 2024-04-26T20:05:49.415 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.415 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.465 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.523 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.523 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164317 s, 0.0 kB/s 2024-04-26T20:05:49.524 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.524 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.573 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.635 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213522 s, 0.0 kB/s 2024-04-26T20:05:49.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.747 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.747 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.747 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245005 s, 0.0 kB/s 2024-04-26T20:05:49.748 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.749 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:49.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:49.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:49.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260666 s, 0.0 kB/s 2024-04-26T20:05:49.907 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:49.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:49.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:49.960 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.018 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.076 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000135748 s, 0.0 kB/s 2024-04-26T20:05:50.078 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.078 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.134 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019365 s, 0.0 kB/s 2024-04-26T20:05:50.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.244 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.350 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158713 s, 0.0 kB/s 2024-04-26T20:05:50.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.351 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191514 s, 0.0 kB/s 2024-04-26T20:05:50.469 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.469 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.520 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000296163 s, 0.0 kB/s 2024-04-26T20:05:50.629 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.629 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.744 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024014 s, 0.0 kB/s 2024-04-26T20:05:50.746 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:50.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:50.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:50.854 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175777 s, 0.0 kB/s 2024-04-26T20:05:50.855 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:50.855 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:50.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:50.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.008 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307108 s, 0.0 kB/s 2024-04-26T20:05:51.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.009 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.122 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232932 s, 0.0 kB/s 2024-04-26T20:05:51.124 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.124 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.234 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184977 s, 0.0 kB/s 2024-04-26T20:05:51.235 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.235 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.287 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180403 s, 0.0 kB/s 2024-04-26T20:05:51.345 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.397 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.455 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021397 s, 0.0 kB/s 2024-04-26T20:05:51.456 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.456 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.508 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.563 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.564 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000141396 s, 0.0 kB/s 2024-04-26T20:05:51.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.613 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184433 s, 0.0 kB/s 2024-04-26T20:05:51.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.718 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.773 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195971 s, 0.0 kB/s 2024-04-26T20:05:51.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:51.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:51.888 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:51.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:51.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:51.948 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000124002 s, 0.0 kB/s 2024-04-26T20:05:51.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:51.949 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.060 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.060 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186983 s, 0.0 kB/s 2024-04-26T20:05:52.061 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.061 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241307 s, 0.0 kB/s 2024-04-26T20:05:52.213 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.213 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.267 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.325 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.325 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.326 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242249 s, 0.0 kB/s 2024-04-26T20:05:52.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.377 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.482 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162098 s, 0.0 kB/s 2024-04-26T20:05:52.483 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.483 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220264 s, 0.0 kB/s 2024-04-26T20:05:52.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.709 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169142 s, 0.0 kB/s 2024-04-26T20:05:52.710 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.710 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.763 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.823 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207312 s, 0.0 kB/s 2024-04-26T20:05:52.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.824 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.880 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:52.943 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236449 s, 0.0 kB/s 2024-04-26T20:05:52.944 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:52.944 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:52.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:52.996 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235669 s, 0.0 kB/s 2024-04-26T20:05:53.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.054 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.106 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.163 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.163 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.163 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258244 s, 0.0 kB/s 2024-04-26T20:05:53.164 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.164 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.217 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196845 s, 0.0 kB/s 2024-04-26T20:05:53.280 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.280 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.336 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.396 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.396 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.396 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000343935 s, 0.0 kB/s 2024-04-26T20:05:53.397 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.397 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185285 s, 0.0 kB/s 2024-04-26T20:05:53.512 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.512 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.561 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229412 s, 0.0 kB/s 2024-04-26T20:05:53.669 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.785 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.785 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220208 s, 0.0 kB/s 2024-04-26T20:05:53.786 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.787 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.834 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:53.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:53.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:53.939 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173104 s, 0.0 kB/s 2024-04-26T20:05:53.940 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:53.940 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:53.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:53.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250095 s, 0.0 kB/s 2024-04-26T20:05:54.055 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.055 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242539 s, 0.0 kB/s 2024-04-26T20:05:54.167 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.168 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.216 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.321 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.321 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.321 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000318338 s, 0.0 kB/s 2024-04-26T20:05:54.322 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.322 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.378 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.440 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271295 s, 0.0 kB/s 2024-04-26T20:05:54.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.559 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246452 s, 0.0 kB/s 2024-04-26T20:05:54.560 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.560 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.615 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.676 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000297999 s, 0.0 kB/s 2024-04-26T20:05:54.677 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.732 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.791 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.791 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.791 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179634 s, 0.0 kB/s 2024-04-26T20:05:54.793 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.793 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.845 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:54.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:54.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:54.905 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275732 s, 0.0 kB/s 2024-04-26T20:05:54.907 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:54.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:54.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:54.959 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.017 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151756 s, 0.0 kB/s 2024-04-26T20:05:55.018 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.133 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024041 s, 0.0 kB/s 2024-04-26T20:05:55.135 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.135 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.246 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166214 s, 0.0 kB/s 2024-04-26T20:05:55.247 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.247 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.300 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.360 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260759 s, 0.0 kB/s 2024-04-26T20:05:55.361 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.361 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.469 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262657 s, 0.0 kB/s 2024-04-26T20:05:55.470 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.470 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.585 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236213 s, 0.0 kB/s 2024-04-26T20:05:55.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.742 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237721 s, 0.0 kB/s 2024-04-26T20:05:55.743 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.743 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221124 s, 0.0 kB/s 2024-04-26T20:05:55.854 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.854 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:55.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:55.907 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:55.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:55.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:55.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145108 s, 0.0 kB/s 2024-04-26T20:05:55.968 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:55.968 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.079 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168481 s, 0.0 kB/s 2024-04-26T20:05:56.080 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.080 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.129 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.234 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209728 s, 0.0 kB/s 2024-04-26T20:05:56.235 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.235 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.287 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.346 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209781 s, 0.0 kB/s 2024-04-26T20:05:56.347 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.347 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.404 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.463 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160039 s, 0.0 kB/s 2024-04-26T20:05:56.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.464 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.515 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.573 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.574 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170412 s, 0.0 kB/s 2024-04-26T20:05:56.575 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.575 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.624 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.729 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253293 s, 0.0 kB/s 2024-04-26T20:05:56.731 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.731 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.787 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:56.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:56.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:56.847 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196808 s, 0.0 kB/s 2024-04-26T20:05:56.848 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:56.848 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:56.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:56.900 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.005 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.005 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.006 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185084 s, 0.0 kB/s 2024-04-26T20:05:57.007 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.007 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.061 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.123 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.123 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.123 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204653 s, 0.0 kB/s 2024-04-26T20:05:57.124 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.124 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.174 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.282 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220528 s, 0.0 kB/s 2024-04-26T20:05:57.283 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.283 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.336 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.397 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000326311 s, 0.0 kB/s 2024-04-26T20:05:57.398 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.511 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196349 s, 0.0 kB/s 2024-04-26T20:05:57.512 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.512 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.565 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.627 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148904 s, 0.0 kB/s 2024-04-26T20:05:57.628 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.629 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.681 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277445 s, 0.0 kB/s 2024-04-26T20:05:57.742 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.742 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.857 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238087 s, 0.0 kB/s 2024-04-26T20:05:57.858 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.858 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:57.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:57.909 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:57.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:57.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:57.974 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207735 s, 0.0 kB/s 2024-04-26T20:05:57.975 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:57.975 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.029 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.091 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.091 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.091 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215844 s, 0.0 kB/s 2024-04-26T20:05:58.092 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.092 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.146 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.205 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.205 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200712 s, 0.0 kB/s 2024-04-26T20:05:58.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.206 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.253 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.358 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275197 s, 0.0 kB/s 2024-04-26T20:05:58.360 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229747 s, 0.0 kB/s 2024-04-26T20:05:58.473 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.473 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.522 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294567 s, 0.0 kB/s 2024-04-26T20:05:58.626 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.626 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.682 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204778 s, 0.0 kB/s 2024-04-26T20:05:58.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.744 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:58.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:58.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:58.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233184 s, 0.0 kB/s 2024-04-26T20:05:58.854 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:58.854 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:58.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:58.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.007 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229671 s, 0.0 kB/s 2024-04-26T20:05:59.008 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.008 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.056 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.165 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223753 s, 0.0 kB/s 2024-04-26T20:05:59.166 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.166 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.222 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259305 s, 0.0 kB/s 2024-04-26T20:05:59.284 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.284 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.398 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.398 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.398 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250968 s, 0.0 kB/s 2024-04-26T20:05:59.399 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002047 s, 0.0 kB/s 2024-04-26T20:05:59.513 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307984 s, 0.0 kB/s 2024-04-26T20:05:59.632 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.633 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.684 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.746 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184926 s, 0.0 kB/s 2024-04-26T20:05:59.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.801 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151041 s, 0.0 kB/s 2024-04-26T20:05:59.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.862 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:05:59.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:05:59.914 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:05:59.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:05:59.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:05:59.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208093 s, 0.0 kB/s 2024-04-26T20:05:59.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:05:59.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.021 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.128 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020032 s, 0.0 kB/s 2024-04-26T20:06:00.129 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.179 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.284 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176722 s, 0.0 kB/s 2024-04-26T20:06:00.285 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.285 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249594 s, 0.0 kB/s 2024-04-26T20:06:00.436 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.436 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.549 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240076 s, 0.0 kB/s 2024-04-26T20:06:00.550 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.551 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.601 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.657 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235994 s, 0.0 kB/s 2024-04-26T20:06:00.658 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.658 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.706 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019989 s, 0.0 kB/s 2024-04-26T20:06:00.812 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.812 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.867 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:00.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:00.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:00.928 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180854 s, 0.0 kB/s 2024-04-26T20:06:00.929 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:00.929 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:00.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:00.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.044 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223423 s, 0.0 kB/s 2024-04-26T20:06:01.045 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.045 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.099 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.160 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030993 s, 0.0 kB/s 2024-04-26T20:06:01.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.161 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.211 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190504 s, 0.0 kB/s 2024-04-26T20:06:01.318 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.318 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.365 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.472 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147443 s, 0.0 kB/s 2024-04-26T20:06:01.473 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.473 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.590 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023453 s, 0.0 kB/s 2024-04-26T20:06:01.591 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.591 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.642 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.704 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.705 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223486 s, 0.0 kB/s 2024-04-26T20:06:01.706 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.760 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.819 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151481 s, 0.0 kB/s 2024-04-26T20:06:01.820 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.820 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:01.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166994 s, 0.0 kB/s 2024-04-26T20:06:01.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:01.928 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:01.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:01.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.046 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234656 s, 0.0 kB/s 2024-04-26T20:06:02.047 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.047 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.102 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.164 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230794 s, 0.0 kB/s 2024-04-26T20:06:02.165 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.165 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.216 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.277 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207837 s, 0.0 kB/s 2024-04-26T20:06:02.278 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.278 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000125302 s, 0.0 kB/s 2024-04-26T20:06:02.393 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.393 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.444 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.501 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184886 s, 0.0 kB/s 2024-04-26T20:06:02.502 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.502 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.614 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155262 s, 0.0 kB/s 2024-04-26T20:06:02.615 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.615 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.665 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.726 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161679 s, 0.0 kB/s 2024-04-26T20:06:02.727 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.728 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.779 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.839 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.839 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.839 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000131582 s, 0.0 kB/s 2024-04-26T20:06:02.840 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.840 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:02.947 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175581 s, 0.0 kB/s 2024-04-26T20:06:02.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:02.948 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:02.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:02.996 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159291 s, 0.0 kB/s 2024-04-26T20:06:03.103 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.103 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.159 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191042 s, 0.0 kB/s 2024-04-26T20:06:03.219 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.219 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.267 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022119 s, 0.0 kB/s 2024-04-26T20:06:03.374 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.429 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.491 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.491 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.491 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194661 s, 0.0 kB/s 2024-04-26T20:06:03.492 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.492 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.546 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.607 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.607 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016412 s, 0.0 kB/s 2024-04-26T20:06:03.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.608 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200342 s, 0.0 kB/s 2024-04-26T20:06:03.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.718 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.772 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270972 s, 0.0 kB/s 2024-04-26T20:06:03.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:03.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:03.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231142 s, 0.0 kB/s 2024-04-26T20:06:03.950 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:03.950 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.003 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.063 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.063 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146486 s, 0.0 kB/s 2024-04-26T20:06:04.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.064 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.113 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.218 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293306 s, 0.0 kB/s 2024-04-26T20:06:04.219 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.219 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.327 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.327 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.327 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186426 s, 0.0 kB/s 2024-04-26T20:06:04.328 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.328 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.380 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.444 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197548 s, 0.0 kB/s 2024-04-26T20:06:04.445 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.445 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.557 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.557 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.557 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000142773 s, 0.0 kB/s 2024-04-26T20:06:04.558 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.558 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.609 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.719 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194391 s, 0.0 kB/s 2024-04-26T20:06:04.720 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.774 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.838 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241077 s, 0.0 kB/s 2024-04-26T20:06:04.839 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.839 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:04.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:04.895 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:04.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:04.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:04.958 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201781 s, 0.0 kB/s 2024-04-26T20:06:04.959 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:04.959 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.013 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261912 s, 0.0 kB/s 2024-04-26T20:06:05.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.184 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229603 s, 0.0 kB/s 2024-04-26T20:06:05.185 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.235 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.341 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.341 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.342 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157509 s, 0.0 kB/s 2024-04-26T20:06:05.343 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.343 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.399 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.465 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.465 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018941 s, 0.0 kB/s 2024-04-26T20:06:05.466 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.466 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.520 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.583 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208116 s, 0.0 kB/s 2024-04-26T20:06:05.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.638 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.698 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.699 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.699 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017946 s, 0.0 kB/s 2024-04-26T20:06:05.700 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.700 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.752 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.860 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272927 s, 0.0 kB/s 2024-04-26T20:06:05.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.862 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:05.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:05.915 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:05.977 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:05.977 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:05.977 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156483 s, 0.0 kB/s 2024-04-26T20:06:05.978 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:05.978 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.031 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.093 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.093 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.093 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277037 s, 0.0 kB/s 2024-04-26T20:06:06.094 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.095 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.150 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.214 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021639 s, 0.0 kB/s 2024-04-26T20:06:06.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.216 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.268 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.375 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.375 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.375 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188954 s, 0.0 kB/s 2024-04-26T20:06:06.376 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.376 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.490 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.491 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.491 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195418 s, 0.0 kB/s 2024-04-26T20:06:06.492 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.492 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.600 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.600 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.600 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222024 s, 0.0 kB/s 2024-04-26T20:06:06.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.601 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.712 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265241 s, 0.0 kB/s 2024-04-26T20:06:06.713 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.714 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.767 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.825 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233712 s, 0.0 kB/s 2024-04-26T20:06:06.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.826 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:06.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:06.876 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:06.982 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248567 s, 0.0 kB/s 2024-04-26T20:06:06.983 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:06.983 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.036 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.103 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000290706 s, 0.0 kB/s 2024-04-26T20:06:07.104 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.104 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.158 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160252 s, 0.0 kB/s 2024-04-26T20:06:07.218 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.218 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.270 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.326 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158404 s, 0.0 kB/s 2024-04-26T20:06:07.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.378 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.484 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200884 s, 0.0 kB/s 2024-04-26T20:06:07.484 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.485 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.639 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185504 s, 0.0 kB/s 2024-04-26T20:06:07.640 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.640 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.691 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278624 s, 0.0 kB/s 2024-04-26T20:06:07.752 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.752 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.804 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.864 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.864 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.864 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239396 s, 0.0 kB/s 2024-04-26T20:06:07.865 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.865 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:07.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:07.917 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:07.976 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:07.976 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:07.976 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186055 s, 0.0 kB/s 2024-04-26T20:06:07.977 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:07.977 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.031 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.097 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275905 s, 0.0 kB/s 2024-04-26T20:06:08.099 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.099 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.153 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.216 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186527 s, 0.0 kB/s 2024-04-26T20:06:08.217 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.217 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.273 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145194 s, 0.0 kB/s 2024-04-26T20:06:08.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.440 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168944 s, 0.0 kB/s 2024-04-26T20:06:08.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.442 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.495 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017777 s, 0.0 kB/s 2024-04-26T20:06:08.554 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.554 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.708 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161454 s, 0.0 kB/s 2024-04-26T20:06:08.709 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.709 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.764 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.822 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249355 s, 0.0 kB/s 2024-04-26T20:06:08.823 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.823 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:08.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:08.872 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:08.975 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024514 s, 0.0 kB/s 2024-04-26T20:06:08.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:08.976 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.028 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180373 s, 0.0 kB/s 2024-04-26T20:06:09.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.142 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177888 s, 0.0 kB/s 2024-04-26T20:06:09.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.204 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199484 s, 0.0 kB/s 2024-04-26T20:06:09.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.360 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.462 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.462 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.462 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189575 s, 0.0 kB/s 2024-04-26T20:06:09.463 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.463 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.515 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235104 s, 0.0 kB/s 2024-04-26T20:06:09.577 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.577 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.687 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239978 s, 0.0 kB/s 2024-04-26T20:06:09.688 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.688 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.741 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258964 s, 0.0 kB/s 2024-04-26T20:06:09.801 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.801 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.853 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:09.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:09.914 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181653 s, 0.0 kB/s 2024-04-26T20:06:09.915 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:09.915 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:09.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:09.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.070 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.070 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.070 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223462 s, 0.0 kB/s 2024-04-26T20:06:10.071 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.071 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258942 s, 0.0 kB/s 2024-04-26T20:06:10.184 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.184 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.236 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293879 s, 0.0 kB/s 2024-04-26T20:06:10.302 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.303 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.356 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.411 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.411 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.411 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172834 s, 0.0 kB/s 2024-04-26T20:06:10.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.412 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.459 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.566 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000142918 s, 0.0 kB/s 2024-04-26T20:06:10.569 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.569 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.622 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.682 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190363 s, 0.0 kB/s 2024-04-26T20:06:10.683 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.683 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.796 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171166 s, 0.0 kB/s 2024-04-26T20:06:10.797 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.797 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.849 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:10.908 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264472 s, 0.0 kB/s 2024-04-26T20:06:10.909 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:10.909 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:10.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:10.962 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.021 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.021 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175612 s, 0.0 kB/s 2024-04-26T20:06:11.022 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.022 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:06:11.169 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:06:11.179 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.179 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.179 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156587 s, 0.0 kB/s 2024-04-26T20:06:11.180 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.180 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.265 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.397 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222842 s, 0.0 kB/s 2024-04-26T20:06:11.398 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.398 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.453 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238771 s, 0.0 kB/s 2024-04-26T20:06:11.522 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.522 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.578 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.635 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00012951 s, 0.0 kB/s 2024-04-26T20:06:11.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.637 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.691 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.752 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164201 s, 0.0 kB/s 2024-04-26T20:06:11.753 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.753 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.807 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.867 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.867 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.868 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171321 s, 0.0 kB/s 2024-04-26T20:06:11.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.869 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:11.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:11.920 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:11.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:11.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:11.980 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215498 s, 0.0 kB/s 2024-04-26T20:06:11.981 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:11.981 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:12.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:12.035 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:12.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:12.245 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000301599 s, 0.0 kB/s 2024-04-26T20:06:12.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:12.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:12.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:12.503 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:12.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:12.728 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242295 s, 0.0 kB/s 2024-04-26T20:06:12.780 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:12.828 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:12.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:12.978 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:13.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:13.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:13.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315504 s, 0.0 kB/s 2024-04-26T20:06:13.260 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:13.362 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:13.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:13.512 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:13.646 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:13.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:13.779 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152837 s, 0.0 kB/s 2024-04-26T20:06:13.849 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:13.912 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:14.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:14.137 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:14.396 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019208 s, 0.0 kB/s 2024-04-26T20:06:14.506 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:14.606 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:14.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:14.763 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:14.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226486 s, 0.0 kB/s 2024-04-26T20:06:15.014 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:15.063 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:15.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:15.213 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:15.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:15.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:15.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267475 s, 0.0 kB/s 2024-04-26T20:06:15.531 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:15.580 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:15.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:15.730 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:15.846 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:15.880 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:15.946 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165576 s, 0.0 kB/s 2024-04-26T20:06:15.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:16.030 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:16.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:16.222 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:16.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190279 s, 0.0 kB/s 2024-04-26T20:06:16.477 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:16.522 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:16.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:16.689 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:16.956 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263194 s, 0.0 kB/s 2024-04-26T20:06:17.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:17.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:17.323 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:17.456 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:17.523 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:17.590 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185584 s, 0.0 kB/s 2024-04-26T20:06:17.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:17.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:17.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:17.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:18.132 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243146 s, 0.0 kB/s 2024-04-26T20:06:18.191 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:18.248 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:18.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:18.457 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:18.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:18.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181866 s, 0.0 kB/s 2024-04-26T20:06:18.766 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:18.834 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:18.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:18.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:19.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:19.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:19.207 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219385 s, 0.0 kB/s 2024-04-26T20:06:19.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:19.357 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:19.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:19.507 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:19.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:19.707 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235659 s, 0.0 kB/s 2024-04-26T20:06:19.759 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:19.824 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:19.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:19.982 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:20.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:20.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:20.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015107 s, 0.0 kB/s 2024-04-26T20:06:20.250 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:20.366 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:20.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:20.524 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:20.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:20.675 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:20.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174234 s, 0.0 kB/s 2024-04-26T20:06:20.784 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:20.863 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:20.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:21.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:21.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:21.275 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151775 s, 0.0 kB/s 2024-04-26T20:06:21.326 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:21.375 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:21.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:21.525 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:21.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:21.675 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:21.725 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022327 s, 0.0 kB/s 2024-04-26T20:06:21.776 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:21.825 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:21.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:21.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:22.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:22.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:22.259 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190809 s, 0.0 kB/s 2024-04-26T20:06:22.317 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:22.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:22.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:22.509 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:22.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000325023 s, 0.0 kB/s 2024-04-26T20:06:22.760 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:22.809 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:22.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:22.984 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:23.226 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171029 s, 0.0 kB/s 2024-04-26T20:06:23.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:23.326 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:23.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:23.476 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:23.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:23.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:23.676 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229012 s, 0.0 kB/s 2024-04-26T20:06:23.776 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:23.804 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:23.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:23.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:24.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:24.220 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244747 s, 0.0 kB/s 2024-04-26T20:06:24.255 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:24.313 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:24.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:24.429 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:24.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:24.651 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231477 s, 0.0 kB/s 2024-04-26T20:06:24.683 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:24.713 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:24.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:24.829 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:24.928 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:25.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:25.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147235 s, 0.0 kB/s 2024-04-26T20:06:25.164 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:25.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:25.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:25.313 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:25.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:25.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:25.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172115 s, 0.0 kB/s 2024-04-26T20:06:25.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:25.588 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:25.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:25.755 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:25.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:25.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:25.964 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254932 s, 0.0 kB/s 2024-04-26T20:06:25.990 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:26.064 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:26.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:26.164 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:26.255 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:26.314 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260452 s, 0.0 kB/s 2024-04-26T20:06:26.340 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:26.356 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:26.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:26.464 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:26.588 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:26.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:26.681 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212775 s, 0.0 kB/s 2024-04-26T20:06:26.699 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:26.715 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:26.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:26.848 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:26.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:26.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:26.982 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022509 s, 0.0 kB/s 2024-04-26T20:06:26.999 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:27.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:27.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:27.115 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:27.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:27.215 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:27.232 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192382 s, 0.0 kB/s 2024-04-26T20:06:27.250 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:27.314 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:27.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:27.440 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195982 s, 0.0 kB/s 2024-04-26T20:06:27.679 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:27.741 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:27.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:27.914 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:27.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:28.033 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222632 s, 0.0 kB/s 2024-04-26T20:06:28.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:28.131 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:28.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:28.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:28.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:28.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:28.479 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258778 s, 0.0 kB/s 2024-04-26T20:06:28.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:28.592 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:28.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:28.761 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:28.858 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:28.892 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:28.925 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200683 s, 0.0 kB/s 2024-04-26T20:06:28.960 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:28.992 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:29.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:29.202 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:29.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:29.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:29.384 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229927 s, 0.0 kB/s 2024-04-26T20:06:29.434 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:29.485 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:29.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:29.618 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:29.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:29.989 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146859 s, 0.0 kB/s 2024-04-26T20:06:30.098 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:30.126 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:30.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:30.260 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:30.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:30.393 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:30.427 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178978 s, 0.0 kB/s 2024-04-26T20:06:30.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:30.567 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:30.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:30.772 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:30.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:30.894 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:30.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247748 s, 0.0 kB/s 2024-04-26T20:06:30.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:30.994 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:31.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:31.119 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:31.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:31.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:31.336 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236401 s, 0.0 kB/s 2024-04-26T20:06:31.370 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:31.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:31.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:31.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:31.627 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:31.644 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:31.678 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315129 s, 0.0 kB/s 2024-04-26T20:06:31.713 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:31.753 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:31.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:31.911 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:31.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:32.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:32.070 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194666 s, 0.0 kB/s 2024-04-26T20:06:32.110 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:32.146 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:32.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:32.236 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:32.323 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:32.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:32.428 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255083 s, 0.0 kB/s 2024-04-26T20:06:32.446 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:32.486 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:32.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:32.612 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:32.695 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:32.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:32.729 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241053 s, 0.0 kB/s 2024-04-26T20:06:32.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:32.854 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:32.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:32.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:33.060 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:33.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:33.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198205 s, 0.0 kB/s 2024-04-26T20:06:33.114 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:33.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:33.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:33.263 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:33.346 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:33.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182976 s, 0.0 kB/s 2024-04-26T20:06:33.439 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:33.488 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:33.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:33.597 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:33.737 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:33.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:33.824 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180889 s, 0.0 kB/s 2024-04-26T20:06:33.848 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:33.897 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:33.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:34.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:34.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:34.189 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:34.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203711 s, 0.0 kB/s 2024-04-26T20:06:34.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:34.323 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:34.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:34.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:34.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:34.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:34.639 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234206 s, 0.0 kB/s 2024-04-26T20:06:34.723 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:34.756 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:34.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:34.873 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:35.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:35.031 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:35.065 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217153 s, 0.0 kB/s 2024-04-26T20:06:35.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:35.206 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:35.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:35.323 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:35.476 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:35.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:35.540 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187911 s, 0.0 kB/s 2024-04-26T20:06:35.575 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:35.607 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:35.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:35.748 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:35.915 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282406 s, 0.0 kB/s 2024-04-26T20:06:36.016 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:36.065 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:36.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:36.174 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:36.280 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:36.392 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194038 s, 0.0 kB/s 2024-04-26T20:06:36.450 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:36.474 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:36.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:36.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:36.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:36.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168661 s, 0.0 kB/s 2024-04-26T20:06:36.867 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:36.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:36.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:37.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:37.116 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:37.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:37.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180716 s, 0.0 kB/s 2024-04-26T20:06:37.217 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:37.250 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:37.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:37.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:37.541 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205836 s, 0.0 kB/s 2024-04-26T20:06:37.593 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:37.671 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:37.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:37.875 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:37.975 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:38.009 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:38.034 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220278 s, 0.0 kB/s 2024-04-26T20:06:38.052 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:38.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:38.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:38.176 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:38.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:38.383 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216327 s, 0.0 kB/s 2024-04-26T20:06:38.411 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:38.434 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:38.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:38.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:38.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:38.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206739 s, 0.0 kB/s 2024-04-26T20:06:38.827 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:38.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:38.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:38.935 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:39.018 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:39.123 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257018 s, 0.0 kB/s 2024-04-26T20:06:39.145 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:39.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:39.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:39.327 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:39.444 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000329002 s, 0.0 kB/s 2024-04-26T20:06:39.462 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:39.536 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:39.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:39.695 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:39.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:39.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:39.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260549 s, 0.0 kB/s 2024-04-26T20:06:39.829 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:39.844 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:39.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:39.961 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:40.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:40.161 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:40.228 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286146 s, 0.0 kB/s 2024-04-26T20:06:40.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:40.295 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:40.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:40.428 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:40.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:40.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:40.665 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262627 s, 0.0 kB/s 2024-04-26T20:06:40.716 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:40.765 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:40.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:40.915 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:41.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:41.132 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:41.182 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167748 s, 0.0 kB/s 2024-04-26T20:06:41.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:41.299 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:41.332 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:06:41.395 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:06:41.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:41.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:41.637 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:41.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:41.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201603 s, 0.0 kB/s 2024-04-26T20:06:41.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:41.874 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:41.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:42.065 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:42.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:42.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:42.274 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257231 s, 0.0 kB/s 2024-04-26T20:06:42.325 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:42.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:42.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:42.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:42.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:42.874 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270138 s, 0.0 kB/s 2024-04-26T20:06:42.925 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:43.015 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:43.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:43.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:43.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:43.324 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:43.374 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187419 s, 0.0 kB/s 2024-04-26T20:06:43.426 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:43.483 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:43.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:43.683 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:43.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:43.850 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149516 s, 0.0 kB/s 2024-04-26T20:06:43.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:43.916 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:44.070 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:44.185 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:44.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:44.392 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178555 s, 0.0 kB/s 2024-04-26T20:06:44.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:44.459 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:44.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:44.585 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:44.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:44.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:44.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164991 s, 0.0 kB/s 2024-04-26T20:06:44.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:44.844 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:44.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:44.976 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:45.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:45.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256252 s, 0.0 kB/s 2024-04-26T20:06:45.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:45.409 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:45.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:45.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:45.793 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155728 s, 0.0 kB/s 2024-04-26T20:06:45.844 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:45.893 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:45.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:46.043 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:46.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:46.310 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189018 s, 0.0 kB/s 2024-04-26T20:06:46.361 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:46.410 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:46.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:46.564 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:46.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:46.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183771 s, 0.0 kB/s 2024-04-26T20:06:46.895 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:46.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:47.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:47.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:47.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:47.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:47.385 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251558 s, 0.0 kB/s 2024-04-26T20:06:47.437 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:47.485 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:47.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:47.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:47.753 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:47.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:47.870 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021313 s, 0.0 kB/s 2024-04-26T20:06:47.912 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:47.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:48.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:48.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:48.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:48.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:48.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159118 s, 0.0 kB/s 2024-04-26T20:06:48.395 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:48.444 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:48.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:48.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:48.844 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:48.938 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220339 s, 0.0 kB/s 2024-04-26T20:06:48.996 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:49.062 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:49.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:49.261 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:49.377 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:49.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:49.487 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176525 s, 0.0 kB/s 2024-04-26T20:06:49.538 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:49.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:49.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:49.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:49.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:49.961 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017174 s, 0.0 kB/s 2024-04-26T20:06:50.096 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:50.145 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:50.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:50.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:50.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:50.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:50.495 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196146 s, 0.0 kB/s 2024-04-26T20:06:50.546 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:50.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:50.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:50.746 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:50.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:50.921 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:50.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243069 s, 0.0 kB/s 2024-04-26T20:06:51.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:51.163 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:51.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:51.313 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:51.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:51.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169949 s, 0.0 kB/s 2024-04-26T20:06:51.598 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:51.646 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:51.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:51.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:51.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:51.963 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:52.013 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206246 s, 0.0 kB/s 2024-04-26T20:06:52.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:52.130 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:52.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:52.263 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:52.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:52.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:52.489 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216969 s, 0.0 kB/s 2024-04-26T20:06:52.523 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:52.555 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:52.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:52.697 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:52.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:52.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:52.889 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151 s, 0.0 kB/s 2024-04-26T20:06:52.933 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:52.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:53.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:53.098 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:53.206 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002014 s, 0.0 kB/s 2024-04-26T20:06:53.335 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:53.375 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:53.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:53.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:53.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:53.747 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155493 s, 0.0 kB/s 2024-04-26T20:06:53.783 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:53.815 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:53.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:53.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:54.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:54.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190488 s, 0.0 kB/s 2024-04-26T20:06:54.183 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:54.215 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:54.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:54.332 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:54.428 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:54.465 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:54.499 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148121 s, 0.0 kB/s 2024-04-26T20:06:54.533 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:54.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:54.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:54.732 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:54.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:54.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:54.957 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172015 s, 0.0 kB/s 2024-04-26T20:06:54.992 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:55.024 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:55.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:55.166 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:55.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:55.291 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214996 s, 0.0 kB/s 2024-04-26T20:06:55.392 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:55.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:55.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:55.608 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:55.724 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:55.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:55.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196112 s, 0.0 kB/s 2024-04-26T20:06:55.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:55.933 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:56.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:56.083 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:56.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:56.300 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184006 s, 0.0 kB/s 2024-04-26T20:06:56.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:56.400 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:56.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:56.578 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:56.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:56.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:56.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205435 s, 0.0 kB/s 2024-04-26T20:06:56.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:56.958 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:57.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:57.117 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:57.231 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:57.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159484 s, 0.0 kB/s 2024-04-26T20:06:57.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:57.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:57.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:57.626 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:57.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:57.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:57.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229669 s, 0.0 kB/s 2024-04-26T20:06:57.885 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:57.934 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:58.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:58.084 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:58.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:58.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:58.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171181 s, 0.0 kB/s 2024-04-26T20:06:58.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:58.443 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:58.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:58.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:58.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:58.793 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016935 s, 0.0 kB/s 2024-04-26T20:06:58.844 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:58.899 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.085 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.218 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154994 s, 0.0 kB/s 2024-04-26T20:06:59.219 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.219 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187556 s, 0.0 kB/s 2024-04-26T20:06:59.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.334 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.384 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000137654 s, 0.0 kB/s 2024-04-26T20:06:59.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.593 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265531 s, 0.0 kB/s 2024-04-26T20:06:59.595 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206819 s, 0.0 kB/s 2024-04-26T20:06:59.707 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.759 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:06:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:06:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:06:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145802 s, 0.0 kB/s 2024-04-26T20:06:59.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:06:59.862 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:06:59.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:06:59.910 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:00.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:00.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:00.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025738 s, 0.0 kB/s 2024-04-26T20:07:00.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:00.013 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:00.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:00.070 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:00.132 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:00.132 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:00.132 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184429 s, 0.0 kB/s 2024-04-26T20:07:00.133 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:00.133 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:00.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:00.188 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:00.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:00.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:00.250 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182907 s, 0.0 kB/s 2024-04-26T20:07:00.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:00.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:00.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:00.301 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246608 s, 0.0 kB/s 2024-04-26T20:07:00.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:00.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:00.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:00.553 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:00.667 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:00.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:00.752 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015885 s, 0.0 kB/s 2024-04-26T20:07:00.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:00.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:00.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:00.994 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:01.144 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:01.194 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173091 s, 0.0 kB/s 2024-04-26T20:07:01.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:01.319 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:01.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:01.452 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:01.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225885 s, 0.0 kB/s 2024-04-26T20:07:01.737 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:01.785 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:01.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:01.919 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:02.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:02.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231845 s, 0.0 kB/s 2024-04-26T20:07:02.196 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:02.244 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:02.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:02.378 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:02.561 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:02.611 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206241 s, 0.0 kB/s 2024-04-26T20:07:02.663 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:02.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:02.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:02.845 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:02.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:03.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:03.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228857 s, 0.0 kB/s 2024-04-26T20:07:03.104 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:03.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:03.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:03.354 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:03.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:03.503 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:03.594 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172923 s, 0.0 kB/s 2024-04-26T20:07:03.646 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:03.695 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:03.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:03.837 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:03.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226033 s, 0.0 kB/s 2024-04-26T20:07:04.122 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:04.170 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:04.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:04.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:04.436 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:04.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206664 s, 0.0 kB/s 2024-04-26T20:07:04.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:04.645 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:04.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:04.796 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:04.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:04.946 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150923 s, 0.0 kB/s 2024-04-26T20:07:05.047 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:05.139 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:05.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:05.308 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:05.429 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:05.546 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208533 s, 0.0 kB/s 2024-04-26T20:07:05.597 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:05.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:05.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:05.796 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:05.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:05.996 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202083 s, 0.0 kB/s 2024-04-26T20:07:06.047 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:06.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:06.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:06.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:06.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:06.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:06.488 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216182 s, 0.0 kB/s 2024-04-26T20:07:06.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:06.588 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:06.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:06.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:06.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:06.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:06.939 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252784 s, 0.0 kB/s 2024-04-26T20:07:06.990 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:07.039 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:07.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:07.189 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:07.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:07.414 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018436 s, 0.0 kB/s 2024-04-26T20:07:07.499 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:07.558 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:07.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:07.706 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:07.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:07.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179046 s, 0.0 kB/s 2024-04-26T20:07:07.957 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:08.027 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:08.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:08.219 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:08.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:08.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:08.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232114 s, 0.0 kB/s 2024-04-26T20:07:08.553 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:08.602 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:08.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:08.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:08.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:08.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:09.056 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023141 s, 0.0 kB/s 2024-04-26T20:07:09.197 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:09.244 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:09.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:09.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:09.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:09.611 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203228 s, 0.0 kB/s 2024-04-26T20:07:09.662 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:09.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:09.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:09.853 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:10.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:10.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184425 s, 0.0 kB/s 2024-04-26T20:07:10.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:10.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:10.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:10.462 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:10.611 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:10.661 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222617 s, 0.0 kB/s 2024-04-26T20:07:10.713 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:10.761 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:10.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:10.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:11.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:11.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:11.112 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223708 s, 0.0 kB/s 2024-04-26T20:07:11.163 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:11.212 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:11.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:11.404 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:11.537 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:11.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:11.620 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184492 s, 0.0 kB/s 2024-04-26T20:07:11.671 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:11.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:11.793 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:07:11.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:11.879 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:11.930 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:07:12.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222752 s, 0.0 kB/s 2024-04-26T20:07:12.189 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:12.237 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:12.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:12.397 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:12.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:12.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:12.660 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208158 s, 0.0 kB/s 2024-04-26T20:07:12.714 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:12.762 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:12.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:12.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:13.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:13.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:13.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271401 s, 0.0 kB/s 2024-04-26T20:07:13.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:13.221 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:13.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:13.363 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:13.480 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:13.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:13.604 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209453 s, 0.0 kB/s 2024-04-26T20:07:13.656 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:13.705 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:13.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:13.888 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:14.005 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:14.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:14.097 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190347 s, 0.0 kB/s 2024-04-26T20:07:14.148 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:14.197 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:14.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:14.339 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:14.489 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:14.539 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222192 s, 0.0 kB/s 2024-04-26T20:07:14.590 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:14.647 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:14.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:14.707 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:14.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:14.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:14.770 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177517 s, 0.0 kB/s 2024-04-26T20:07:14.771 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:14.771 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:14.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:14.823 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218344 s, 0.0 kB/s 2024-04-26T20:07:14.885 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:14.885 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:14.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:14.937 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:14.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:14.993 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:14.994 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171714 s, 0.0 kB/s 2024-04-26T20:07:14.995 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:14.995 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.046 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.108 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269196 s, 0.0 kB/s 2024-04-26T20:07:15.109 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.109 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.162 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191166 s, 0.0 kB/s 2024-04-26T20:07:15.225 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.225 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.277 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000305495 s, 0.0 kB/s 2024-04-26T20:07:15.334 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.334 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237374 s, 0.0 kB/s 2024-04-26T20:07:15.447 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.503 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.563 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.563 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177628 s, 0.0 kB/s 2024-04-26T20:07:15.564 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.617 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.683 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000324177 s, 0.0 kB/s 2024-04-26T20:07:15.684 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.684 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190002 s, 0.0 kB/s 2024-04-26T20:07:15.801 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.801 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:15.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:15.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:15.919 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019625 s, 0.0 kB/s 2024-04-26T20:07:15.920 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:15.920 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:15.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:15.972 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225352 s, 0.0 kB/s 2024-04-26T20:07:16.034 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.035 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.090 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.150 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255439 s, 0.0 kB/s 2024-04-26T20:07:16.151 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.151 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.267 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197104 s, 0.0 kB/s 2024-04-26T20:07:16.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.314 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.419 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.419 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178562 s, 0.0 kB/s 2024-04-26T20:07:16.420 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.421 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.475 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.534 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155614 s, 0.0 kB/s 2024-04-26T20:07:16.535 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.535 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.583 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.689 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.689 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000137033 s, 0.0 kB/s 2024-04-26T20:07:16.690 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.690 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.740 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233585 s, 0.0 kB/s 2024-04-26T20:07:16.800 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.800 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:16.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:16.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:16.915 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186519 s, 0.0 kB/s 2024-04-26T20:07:16.916 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:16.916 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:16.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:16.965 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.026 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.027 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145094 s, 0.0 kB/s 2024-04-26T20:07:17.028 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.133 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145346 s, 0.0 kB/s 2024-04-26T20:07:17.134 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.134 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210822 s, 0.0 kB/s 2024-04-26T20:07:17.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.304 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.366 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.366 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.366 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206653 s, 0.0 kB/s 2024-04-26T20:07:17.367 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.367 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.421 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.484 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.484 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.484 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246274 s, 0.0 kB/s 2024-04-26T20:07:17.485 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.486 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.600 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.600 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.601 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263328 s, 0.0 kB/s 2024-04-26T20:07:17.602 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.602 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.654 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256123 s, 0.0 kB/s 2024-04-26T20:07:17.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.718 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262203 s, 0.0 kB/s 2024-04-26T20:07:17.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:17.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:17.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:17.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:17.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:17.955 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147874 s, 0.0 kB/s 2024-04-26T20:07:17.956 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:17.956 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.005 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.112 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274674 s, 0.0 kB/s 2024-04-26T20:07:18.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.113 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.165 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.225 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260977 s, 0.0 kB/s 2024-04-26T20:07:18.226 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.226 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.273 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235417 s, 0.0 kB/s 2024-04-26T20:07:18.380 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.487 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192542 s, 0.0 kB/s 2024-04-26T20:07:18.488 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.488 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.641 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.641 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.641 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020606 s, 0.0 kB/s 2024-04-26T20:07:18.642 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.642 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.694 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195376 s, 0.0 kB/s 2024-04-26T20:07:18.757 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.757 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.807 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.861 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019975 s, 0.0 kB/s 2024-04-26T20:07:18.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.862 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:18.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:18.913 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217236 s, 0.0 kB/s 2024-04-26T20:07:18.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:18.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174604 s, 0.0 kB/s 2024-04-26T20:07:19.083 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.083 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.135 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.194 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174372 s, 0.0 kB/s 2024-04-26T20:07:19.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179141 s, 0.0 kB/s 2024-04-26T20:07:19.308 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.360 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.464 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238013 s, 0.0 kB/s 2024-04-26T20:07:19.466 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.466 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.518 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255569 s, 0.0 kB/s 2024-04-26T20:07:19.579 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.579 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.630 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.688 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000121033 s, 0.0 kB/s 2024-04-26T20:07:19.689 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.689 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.842 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215975 s, 0.0 kB/s 2024-04-26T20:07:19.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:19.897 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:19.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:19.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:19.954 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000139165 s, 0.0 kB/s 2024-04-26T20:07:19.955 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:19.955 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.011 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.067 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.068 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.068 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164539 s, 0.0 kB/s 2024-04-26T20:07:20.069 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.069 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.120 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.178 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022023 s, 0.0 kB/s 2024-04-26T20:07:20.179 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.232 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.289 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249339 s, 0.0 kB/s 2024-04-26T20:07:20.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.291 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.340 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.445 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200457 s, 0.0 kB/s 2024-04-26T20:07:20.446 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.446 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.562 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192033 s, 0.0 kB/s 2024-04-26T20:07:20.564 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.678 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151117 s, 0.0 kB/s 2024-04-26T20:07:20.679 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.679 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.728 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.829 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160553 s, 0.0 kB/s 2024-04-26T20:07:20.831 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.831 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:20.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:20.884 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:20.983 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:20.983 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:20.983 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000141209 s, 0.0 kB/s 2024-04-26T20:07:20.984 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:20.984 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.031 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.134 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.134 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.134 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186112 s, 0.0 kB/s 2024-04-26T20:07:21.135 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.135 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.190 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.250 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266165 s, 0.0 kB/s 2024-04-26T20:07:21.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.305 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000381275 s, 0.0 kB/s 2024-04-26T20:07:21.369 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.369 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.419 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.522 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016766 s, 0.0 kB/s 2024-04-26T20:07:21.523 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.523 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.575 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157196 s, 0.0 kB/s 2024-04-26T20:07:21.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.684 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150648 s, 0.0 kB/s 2024-04-26T20:07:21.790 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.790 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.842 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:21.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:21.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:21.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261661 s, 0.0 kB/s 2024-04-26T20:07:21.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:21.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:21.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:21.955 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.017 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222168 s, 0.0 kB/s 2024-04-26T20:07:22.018 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.071 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165237 s, 0.0 kB/s 2024-04-26T20:07:22.131 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.131 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.245 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233746 s, 0.0 kB/s 2024-04-26T20:07:22.246 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.246 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.299 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189931 s, 0.0 kB/s 2024-04-26T20:07:22.361 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.361 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.473 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177174 s, 0.0 kB/s 2024-04-26T20:07:22.474 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.474 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.527 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198586 s, 0.0 kB/s 2024-04-26T20:07:22.587 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.587 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.638 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.697 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150753 s, 0.0 kB/s 2024-04-26T20:07:22.699 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.699 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.753 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147631 s, 0.0 kB/s 2024-04-26T20:07:22.815 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.815 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.866 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:22.926 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169956 s, 0.0 kB/s 2024-04-26T20:07:22.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:22.928 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:22.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:22.983 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194789 s, 0.0 kB/s 2024-04-26T20:07:23.043 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.043 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.094 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.198 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201955 s, 0.0 kB/s 2024-04-26T20:07:23.199 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.200 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.306 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.306 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177322 s, 0.0 kB/s 2024-04-26T20:07:23.307 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.355 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.460 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307979 s, 0.0 kB/s 2024-04-26T20:07:23.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.461 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.513 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.575 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.575 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.576 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168613 s, 0.0 kB/s 2024-04-26T20:07:23.577 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.577 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.631 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.694 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158332 s, 0.0 kB/s 2024-04-26T20:07:23.695 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.695 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.745 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.809 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000324514 s, 0.0 kB/s 2024-04-26T20:07:23.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.811 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:23.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:23.861 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:23.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:23.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:23.966 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144636 s, 0.0 kB/s 2024-04-26T20:07:23.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:23.967 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.018 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.076 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213115 s, 0.0 kB/s 2024-04-26T20:07:24.077 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.077 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.130 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190148 s, 0.0 kB/s 2024-04-26T20:07:24.193 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.193 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.304 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017859 s, 0.0 kB/s 2024-04-26T20:07:24.305 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.305 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.359 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150175 s, 0.0 kB/s 2024-04-26T20:07:24.419 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.419 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.470 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170177 s, 0.0 kB/s 2024-04-26T20:07:24.530 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.530 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.580 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.639 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243148 s, 0.0 kB/s 2024-04-26T20:07:24.640 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.640 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.794 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019225 s, 0.0 kB/s 2024-04-26T20:07:24.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.795 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.847 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:24.907 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:24.907 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:24.907 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226839 s, 0.0 kB/s 2024-04-26T20:07:24.908 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:24.908 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:24.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:24.960 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.017 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213344 s, 0.0 kB/s 2024-04-26T20:07:25.018 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.071 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.131 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180192 s, 0.0 kB/s 2024-04-26T20:07:25.132 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.133 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.179 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.287 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020891 s, 0.0 kB/s 2024-04-26T20:07:25.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.288 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.405 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272872 s, 0.0 kB/s 2024-04-26T20:07:25.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.460 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.521 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153008 s, 0.0 kB/s 2024-04-26T20:07:25.522 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.522 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.575 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.630 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219816 s, 0.0 kB/s 2024-04-26T20:07:25.632 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.632 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.685 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158282 s, 0.0 kB/s 2024-04-26T20:07:25.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.744 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:25.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:25.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:25.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307896 s, 0.0 kB/s 2024-04-26T20:07:25.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:25.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:25.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:25.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.010 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196562 s, 0.0 kB/s 2024-04-26T20:07:26.011 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.011 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.121 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154306 s, 0.0 kB/s 2024-04-26T20:07:26.122 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.123 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.230 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213497 s, 0.0 kB/s 2024-04-26T20:07:26.232 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.232 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.285 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.345 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.346 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000129955 s, 0.0 kB/s 2024-04-26T20:07:26.347 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.399 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202941 s, 0.0 kB/s 2024-04-26T20:07:26.457 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.457 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.511 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197467 s, 0.0 kB/s 2024-04-26T20:07:26.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.572 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.625 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.684 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271967 s, 0.0 kB/s 2024-04-26T20:07:26.685 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.685 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.796 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169655 s, 0.0 kB/s 2024-04-26T20:07:26.797 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.797 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.848 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:26.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235747 s, 0.0 kB/s 2024-04-26T20:07:26.907 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:26.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:26.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:26.962 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.020 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195312 s, 0.0 kB/s 2024-04-26T20:07:27.021 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.021 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.073 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204142 s, 0.0 kB/s 2024-04-26T20:07:27.135 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.135 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190586 s, 0.0 kB/s 2024-04-26T20:07:27.247 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.247 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.300 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.361 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190628 s, 0.0 kB/s 2024-04-26T20:07:27.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.362 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.411 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154152 s, 0.0 kB/s 2024-04-26T20:07:27.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.565 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.624 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.624 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148551 s, 0.0 kB/s 2024-04-26T20:07:27.625 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.625 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.676 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021448 s, 0.0 kB/s 2024-04-26T20:07:27.734 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.787 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.847 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230284 s, 0.0 kB/s 2024-04-26T20:07:27.848 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:27.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:27.901 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:27.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:27.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:27.959 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151557 s, 0.0 kB/s 2024-04-26T20:07:27.960 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:27.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.075 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247115 s, 0.0 kB/s 2024-04-26T20:07:28.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.127 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.187 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.188 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163092 s, 0.0 kB/s 2024-04-26T20:07:28.189 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.237 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291121 s, 0.0 kB/s 2024-04-26T20:07:28.345 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.397 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.457 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251953 s, 0.0 kB/s 2024-04-26T20:07:28.458 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.458 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.509 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.566 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.566 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211999 s, 0.0 kB/s 2024-04-26T20:07:28.567 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.567 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.621 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.678 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208861 s, 0.0 kB/s 2024-04-26T20:07:28.679 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.679 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.732 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.799 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185358 s, 0.0 kB/s 2024-04-26T20:07:28.800 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.800 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:28.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:28.851 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:28.952 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:28.952 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:28.953 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000288342 s, 0.0 kB/s 2024-04-26T20:07:28.954 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:28.954 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.002 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.108 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186647 s, 0.0 kB/s 2024-04-26T20:07:29.109 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.109 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.161 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.269 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218463 s, 0.0 kB/s 2024-04-26T20:07:29.270 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.270 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.320 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.380 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247121 s, 0.0 kB/s 2024-04-26T20:07:29.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.381 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.499 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000329177 s, 0.0 kB/s 2024-04-26T20:07:29.500 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.500 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.554 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.615 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220837 s, 0.0 kB/s 2024-04-26T20:07:29.616 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.616 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.670 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.728 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210562 s, 0.0 kB/s 2024-04-26T20:07:29.728 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.729 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.780 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.843 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291314 s, 0.0 kB/s 2024-04-26T20:07:29.844 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.844 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:29.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:29.899 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:29.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:29.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:29.962 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184004 s, 0.0 kB/s 2024-04-26T20:07:29.963 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:29.963 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.015 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147612 s, 0.0 kB/s 2024-04-26T20:07:30.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.124 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.177 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000133349 s, 0.0 kB/s 2024-04-26T20:07:30.178 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.229 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.334 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.334 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000137853 s, 0.0 kB/s 2024-04-26T20:07:30.335 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.335 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.383 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.446 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187526 s, 0.0 kB/s 2024-04-26T20:07:30.447 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.502 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.563 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.563 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207195 s, 0.0 kB/s 2024-04-26T20:07:30.564 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.677 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255999 s, 0.0 kB/s 2024-04-26T20:07:30.678 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.678 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.731 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.789 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238769 s, 0.0 kB/s 2024-04-26T20:07:30.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.791 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.846 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:30.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294694 s, 0.0 kB/s 2024-04-26T20:07:30.907 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:30.908 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:30.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:30.961 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.018 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.018 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.018 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193453 s, 0.0 kB/s 2024-04-26T20:07:31.019 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.019 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.074 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279261 s, 0.0 kB/s 2024-04-26T20:07:31.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.191 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252789 s, 0.0 kB/s 2024-04-26T20:07:31.250 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.250 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.303 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.364 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239574 s, 0.0 kB/s 2024-04-26T20:07:31.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.365 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.517 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.517 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.517 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263582 s, 0.0 kB/s 2024-04-26T20:07:31.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.634 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017714 s, 0.0 kB/s 2024-04-26T20:07:31.635 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.687 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.795 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184701 s, 0.0 kB/s 2024-04-26T20:07:31.796 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.796 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.850 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:31.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:31.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:31.910 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265752 s, 0.0 kB/s 2024-04-26T20:07:31.911 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:31.911 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:31.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:31.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.027 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000143216 s, 0.0 kB/s 2024-04-26T20:07:32.028 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017017 s, 0.0 kB/s 2024-04-26T20:07:32.183 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.183 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.240 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.294 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167123 s, 0.0 kB/s 2024-04-26T20:07:32.295 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.295 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.339 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.443 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.443 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.443 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182578 s, 0.0 kB/s 2024-04-26T20:07:32.444 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.445 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.495 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.564 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179918 s, 0.0 kB/s 2024-04-26T20:07:32.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.619 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.680 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00035012 s, 0.0 kB/s 2024-04-26T20:07:32.682 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.682 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.739 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.798 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148822 s, 0.0 kB/s 2024-04-26T20:07:32.799 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.799 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:32.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:32.850 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:32.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:32.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:32.957 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238913 s, 0.0 kB/s 2024-04-26T20:07:32.958 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:32.958 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.010 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210584 s, 0.0 kB/s 2024-04-26T20:07:33.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.127 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.188 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219856 s, 0.0 kB/s 2024-04-26T20:07:33.189 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.304 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150959 s, 0.0 kB/s 2024-04-26T20:07:33.305 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.305 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.357 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.415 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.415 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149905 s, 0.0 kB/s 2024-04-26T20:07:33.416 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.416 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.463 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197006 s, 0.0 kB/s 2024-04-26T20:07:33.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.627 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.687 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161754 s, 0.0 kB/s 2024-04-26T20:07:33.688 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.688 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.742 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.804 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000139111 s, 0.0 kB/s 2024-04-26T20:07:33.805 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:33.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:33.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:33.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:33.955 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:33.955 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167146 s, 0.0 kB/s 2024-04-26T20:07:33.956 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:33.956 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.008 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257893 s, 0.0 kB/s 2024-04-26T20:07:34.067 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.067 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.120 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204875 s, 0.0 kB/s 2024-04-26T20:07:34.184 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.235 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.340 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188704 s, 0.0 kB/s 2024-04-26T20:07:34.342 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.342 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.392 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.448 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.448 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194588 s, 0.0 kB/s 2024-04-26T20:07:34.449 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.604 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248127 s, 0.0 kB/s 2024-04-26T20:07:34.605 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.605 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.659 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.718 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258782 s, 0.0 kB/s 2024-04-26T20:07:34.719 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.775 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175378 s, 0.0 kB/s 2024-04-26T20:07:34.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.835 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.885 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:34.942 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:34.942 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:34.942 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000133734 s, 0.0 kB/s 2024-04-26T20:07:34.943 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:34.943 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:34.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:34.998 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163383 s, 0.0 kB/s 2024-04-26T20:07:35.058 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.058 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.214 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186183 s, 0.0 kB/s 2024-04-26T20:07:35.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.215 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222506 s, 0.0 kB/s 2024-04-26T20:07:35.334 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.335 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.388 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160916 s, 0.0 kB/s 2024-04-26T20:07:35.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.448 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.568 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.568 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.569 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202868 s, 0.0 kB/s 2024-04-26T20:07:35.570 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.570 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.621 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.680 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.680 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.680 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230364 s, 0.0 kB/s 2024-04-26T20:07:35.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.681 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.733 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.794 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224516 s, 0.0 kB/s 2024-04-26T20:07:35.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.796 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.848 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:35.907 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:35.907 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:35.907 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257675 s, 0.0 kB/s 2024-04-26T20:07:35.908 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:35.908 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:35.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:35.958 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.065 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.066 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219561 s, 0.0 kB/s 2024-04-26T20:07:36.067 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.067 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.119 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.177 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169053 s, 0.0 kB/s 2024-04-26T20:07:36.178 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.226 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.331 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217329 s, 0.0 kB/s 2024-04-26T20:07:36.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.448 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.448 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.448 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000124856 s, 0.0 kB/s 2024-04-26T20:07:36.449 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.603 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.604 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.604 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268146 s, 0.0 kB/s 2024-04-26T20:07:36.605 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.606 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.657 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211566 s, 0.0 kB/s 2024-04-26T20:07:36.718 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.718 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.772 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199657 s, 0.0 kB/s 2024-04-26T20:07:36.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.834 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:36.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:36.882 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:36.986 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:36.987 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:36.987 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234563 s, 0.0 kB/s 2024-04-26T20:07:36.988 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:36.988 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.044 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.104 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.104 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.104 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213022 s, 0.0 kB/s 2024-04-26T20:07:37.105 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.105 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.157 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.218 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000301942 s, 0.0 kB/s 2024-04-26T20:07:37.219 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.220 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.272 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.335 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176026 s, 0.0 kB/s 2024-04-26T20:07:37.336 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.336 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.490 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.490 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.490 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280924 s, 0.0 kB/s 2024-04-26T20:07:37.491 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.491 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.544 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.606 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163064 s, 0.0 kB/s 2024-04-26T20:07:37.610 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.610 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257005 s, 0.0 kB/s 2024-04-26T20:07:37.722 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.722 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.778 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.837 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.838 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187027 s, 0.0 kB/s 2024-04-26T20:07:37.839 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.839 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:37.893 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203182 s, 0.0 kB/s 2024-04-26T20:07:37.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:37.952 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.106 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257251 s, 0.0 kB/s 2024-04-26T20:07:38.107 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.107 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.162 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237342 s, 0.0 kB/s 2024-04-26T20:07:38.226 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.226 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.279 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.339 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156095 s, 0.0 kB/s 2024-04-26T20:07:38.340 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.392 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.454 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.454 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167044 s, 0.0 kB/s 2024-04-26T20:07:38.455 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.455 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.509 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.570 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237929 s, 0.0 kB/s 2024-04-26T20:07:38.571 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.571 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.623 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.678 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014479 s, 0.0 kB/s 2024-04-26T20:07:38.679 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.679 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.730 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000144319 s, 0.0 kB/s 2024-04-26T20:07:38.790 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.790 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.841 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:38.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:38.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:38.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261696 s, 0.0 kB/s 2024-04-26T20:07:38.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:38.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:38.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:38.957 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.019 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207488 s, 0.0 kB/s 2024-04-26T20:07:39.020 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.020 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.073 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015785 s, 0.0 kB/s 2024-04-26T20:07:39.134 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.134 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169101 s, 0.0 kB/s 2024-04-26T20:07:39.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.245 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245245 s, 0.0 kB/s 2024-04-26T20:07:39.402 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.452 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.570 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240499 s, 0.0 kB/s 2024-04-26T20:07:39.571 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.571 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.693 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.693 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.693 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244372 s, 0.0 kB/s 2024-04-26T20:07:39.694 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.745 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.804 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228048 s, 0.0 kB/s 2024-04-26T20:07:39.805 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:39.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:39.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191471 s, 0.0 kB/s 2024-04-26T20:07:39.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:39.961 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.013 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.073 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248791 s, 0.0 kB/s 2024-04-26T20:07:40.074 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.131 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000300061 s, 0.0 kB/s 2024-04-26T20:07:40.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.252 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.308 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.308 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.308 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146524 s, 0.0 kB/s 2024-04-26T20:07:40.309 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.309 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.356 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.460 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.460 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196716 s, 0.0 kB/s 2024-04-26T20:07:40.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.461 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.513 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.618 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150745 s, 0.0 kB/s 2024-04-26T20:07:40.619 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.619 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.676 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.737 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178658 s, 0.0 kB/s 2024-04-26T20:07:40.737 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.738 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.792 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.851 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.851 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.851 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167404 s, 0.0 kB/s 2024-04-26T20:07:40.852 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:40.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:40.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019477 s, 0.0 kB/s 2024-04-26T20:07:40.966 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:40.966 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.021 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.083 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.083 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218817 s, 0.0 kB/s 2024-04-26T20:07:41.084 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.084 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.138 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.195 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159684 s, 0.0 kB/s 2024-04-26T20:07:41.196 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.197 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.309 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00028021 s, 0.0 kB/s 2024-04-26T20:07:41.311 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.364 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263794 s, 0.0 kB/s 2024-04-26T20:07:41.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.539 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.539 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.539 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168114 s, 0.0 kB/s 2024-04-26T20:07:41.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.540 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.647 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151037 s, 0.0 kB/s 2024-04-26T20:07:41.648 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.648 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.696 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191629 s, 0.0 kB/s 2024-04-26T20:07:41.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.803 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:41.916 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240634 s, 0.0 kB/s 2024-04-26T20:07:41.917 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:41.917 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:41.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:41.970 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.028 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150825 s, 0.0 kB/s 2024-04-26T20:07:42.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.082 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.140 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170029 s, 0.0 kB/s 2024-04-26T20:07:42.141 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.141 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.190 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:07:42.194 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:07:42.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.196 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279891 s, 0.0 kB/s 2024-04-26T20:07:42.287 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.287 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175648 s, 0.0 kB/s 2024-04-26T20:07:42.435 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.435 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000301757 s, 0.0 kB/s 2024-04-26T20:07:42.549 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.549 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.600 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241461 s, 0.0 kB/s 2024-04-26T20:07:42.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.741 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.803 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.803 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.803 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178571 s, 0.0 kB/s 2024-04-26T20:07:42.804 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.804 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.860 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:42.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:42.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:42.919 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000134411 s, 0.0 kB/s 2024-04-26T20:07:42.920 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:42.920 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:42.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:42.968 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.073 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150824 s, 0.0 kB/s 2024-04-26T20:07:43.074 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.074 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.124 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.184 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.184 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025111 s, 0.0 kB/s 2024-04-26T20:07:43.185 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.237 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.295 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201186 s, 0.0 kB/s 2024-04-26T20:07:43.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.296 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.344 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.449 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.449 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.449 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194508 s, 0.0 kB/s 2024-04-26T20:07:43.451 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.451 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.505 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.572 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.572 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000303792 s, 0.0 kB/s 2024-04-26T20:07:43.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.627 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.684 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190039 s, 0.0 kB/s 2024-04-26T20:07:43.685 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.685 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.797 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258531 s, 0.0 kB/s 2024-04-26T20:07:43.798 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.798 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:43.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:43.846 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:43.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:43.952 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:43.952 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138741 s, 0.0 kB/s 2024-04-26T20:07:43.953 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:43.953 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.004 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200326 s, 0.0 kB/s 2024-04-26T20:07:44.065 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.065 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.117 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.174 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212334 s, 0.0 kB/s 2024-04-26T20:07:44.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.175 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.226 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.331 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264755 s, 0.0 kB/s 2024-04-26T20:07:44.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.446 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148306 s, 0.0 kB/s 2024-04-26T20:07:44.447 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.552 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.552 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.552 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201731 s, 0.0 kB/s 2024-04-26T20:07:44.553 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.707 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001437 s, 0.0 kB/s 2024-04-26T20:07:44.708 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.708 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.755 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:44.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:44.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:44.857 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206746 s, 0.0 kB/s 2024-04-26T20:07:44.858 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:44.858 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:44.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:44.906 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240152 s, 0.0 kB/s 2024-04-26T20:07:45.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.067 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020341 s, 0.0 kB/s 2024-04-26T20:07:45.131 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.131 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.183 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.242 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.242 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.242 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017239 s, 0.0 kB/s 2024-04-26T20:07:45.243 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.243 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172041 s, 0.0 kB/s 2024-04-26T20:07:45.402 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.515 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256483 s, 0.0 kB/s 2024-04-26T20:07:45.516 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.516 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.571 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176043 s, 0.0 kB/s 2024-04-26T20:07:45.634 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.634 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.746 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000319816 s, 0.0 kB/s 2024-04-26T20:07:45.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.796 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:45.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:45.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:45.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250263 s, 0.0 kB/s 2024-04-26T20:07:45.903 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:45.903 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:45.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:45.957 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184719 s, 0.0 kB/s 2024-04-26T20:07:46.017 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.017 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.067 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180559 s, 0.0 kB/s 2024-04-26T20:07:46.130 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.130 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.181 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.286 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165513 s, 0.0 kB/s 2024-04-26T20:07:46.287 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.287 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000145347 s, 0.0 kB/s 2024-04-26T20:07:46.392 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.392 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.443 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170313 s, 0.0 kB/s 2024-04-26T20:07:46.549 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.549 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.604 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.662 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.662 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.662 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024172 s, 0.0 kB/s 2024-04-26T20:07:46.663 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.663 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.714 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.779 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.779 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.779 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209198 s, 0.0 kB/s 2024-04-26T20:07:46.781 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.837 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:46.898 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:46.899 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207332 s, 0.0 kB/s 2024-04-26T20:07:46.900 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:46.900 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:46.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:46.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:47.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:47.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:47.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261561 s, 0.0 kB/s 2024-04-26T20:07:47.014 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:47.014 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:47.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.063 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:47.124 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:47.125 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:47.125 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235008 s, 0.0 kB/s 2024-04-26T20:07:47.126 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:47.126 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:47.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.178 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153358 s, 0.0 kB/s 2024-04-26T20:07:47.238 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:47.238 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:47.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.290 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:47.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:47.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:47.350 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223111 s, 0.0 kB/s 2024-04-26T20:07:47.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:47.351 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:47.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:47.575 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:47.610 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022016 s, 0.0 kB/s 2024-04-26T20:07:47.711 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:47.768 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:47.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:47.935 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:48.049 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:48.170 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:48.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186823 s, 0.0 kB/s 2024-04-26T20:07:48.278 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:48.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:48.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:48.468 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:48.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:48.635 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:48.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220573 s, 0.0 kB/s 2024-04-26T20:07:48.736 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:48.785 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:48.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:48.927 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:49.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:49.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:49.152 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223163 s, 0.0 kB/s 2024-04-26T20:07:49.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:49.253 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:49.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:49.394 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:49.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:49.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:49.594 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026462 s, 0.0 kB/s 2024-04-26T20:07:49.645 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:49.728 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:49.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:49.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:49.986 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:50.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:50.086 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205974 s, 0.0 kB/s 2024-04-26T20:07:50.150 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:50.232 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:50.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:50.378 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:50.504 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:50.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:50.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195773 s, 0.0 kB/s 2024-04-26T20:07:50.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:50.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:50.829 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:50.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:51.028 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196141 s, 0.0 kB/s 2024-04-26T20:07:51.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:51.128 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:51.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:51.270 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:51.421 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:51.453 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200343 s, 0.0 kB/s 2024-04-26T20:07:51.555 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:51.603 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:51.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:51.753 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:51.872 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:51.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:51.962 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156744 s, 0.0 kB/s 2024-04-26T20:07:52.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:52.062 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:52.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:52.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:52.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:52.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:52.445 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000344325 s, 0.0 kB/s 2024-04-26T20:07:52.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:52.546 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:52.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:52.687 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:52.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:52.837 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:52.921 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162297 s, 0.0 kB/s 2024-04-26T20:07:52.980 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:53.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:53.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:53.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:53.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:53.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000298111 s, 0.0 kB/s 2024-04-26T20:07:53.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:53.555 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:53.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:53.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:53.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:53.913 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:53.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020925 s, 0.0 kB/s 2024-04-26T20:07:54.014 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:54.063 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:54.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:54.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:54.319 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:54.355 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242806 s, 0.0 kB/s 2024-04-26T20:07:54.498 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:54.547 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:54.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:54.722 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:54.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:54.905 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:54.958 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253661 s, 0.0 kB/s 2024-04-26T20:07:55.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:55.075 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:55.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:55.222 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:55.337 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:55.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:55.422 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237677 s, 0.0 kB/s 2024-04-26T20:07:55.490 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:55.539 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:55.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:55.722 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:55.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:55.872 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:55.939 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185568 s, 0.0 kB/s 2024-04-26T20:07:55.990 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:56.063 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:56.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:56.206 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:56.319 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:56.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251273 s, 0.0 kB/s 2024-04-26T20:07:56.449 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:56.514 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:56.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:56.664 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:56.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:56.814 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:56.864 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204171 s, 0.0 kB/s 2024-04-26T20:07:56.916 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:56.965 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:57.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:57.140 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:57.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:57.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:57.381 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024 s, 0.0 kB/s 2024-04-26T20:07:57.432 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:57.481 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:57.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:57.632 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:57.744 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:57.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:57.832 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216533 s, 0.0 kB/s 2024-04-26T20:07:57.933 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:57.982 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:58.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:58.132 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:58.247 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:58.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:58.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150261 s, 0.0 kB/s 2024-04-26T20:07:58.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:58.432 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:58.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:58.582 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:58.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:58.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:58.844 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194866 s, 0.0 kB/s 2024-04-26T20:07:58.892 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:58.941 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:59.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:59.133 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:59.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:59.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240455 s, 0.0 kB/s 2024-04-26T20:07:59.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:59.433 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:07:59.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:07:59.592 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:07:59.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:07:59.783 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:07:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241304 s, 0.0 kB/s 2024-04-26T20:07:59.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:07:59.933 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:00.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:00.083 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:00.211 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:00.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197822 s, 0.0 kB/s 2024-04-26T20:08:00.393 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:00.442 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:00.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:00.642 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:00.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:00.792 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:00.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022197 s, 0.0 kB/s 2024-04-26T20:08:00.893 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:00.942 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:01.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:01.078 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:01.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:01.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:01.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199126 s, 0.0 kB/s 2024-04-26T20:08:01.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:01.376 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:01.584 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:01.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:01.749 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:01.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218403 s, 0.0 kB/s 2024-04-26T20:08:01.861 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:01.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:02.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:02.060 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:02.268 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:02.318 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179165 s, 0.0 kB/s 2024-04-26T20:08:02.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:02.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:02.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:02.660 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:02.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:02.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161147 s, 0.0 kB/s 2024-04-26T20:08:02.911 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:02.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:03.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:03.148 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:03.268 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:03.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:03.377 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233334 s, 0.0 kB/s 2024-04-26T20:08:03.455 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:03.511 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:03.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:03.669 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:03.783 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:03.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159504 s, 0.0 kB/s 2024-04-26T20:08:03.920 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:03.985 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:04.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:04.127 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:04.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:04.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153146 s, 0.0 kB/s 2024-04-26T20:08:04.471 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:04.526 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:04.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:04.669 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:04.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:04.828 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:04.878 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184412 s, 0.0 kB/s 2024-04-26T20:08:04.929 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:04.995 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:05.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:05.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:05.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:05.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:05.461 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268215 s, 0.0 kB/s 2024-04-26T20:08:05.529 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:05.603 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:05.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:05.812 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:06.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:06.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200233 s, 0.0 kB/s 2024-04-26T20:08:06.147 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:06.212 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:06.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:06.387 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:06.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:06.637 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:06.695 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272115 s, 0.0 kB/s 2024-04-26T20:08:06.755 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:06.804 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:06.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:06.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:07.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:07.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:07.179 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229842 s, 0.0 kB/s 2024-04-26T20:08:07.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:07.287 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:07.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:07.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:07.646 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165649 s, 0.0 kB/s 2024-04-26T20:08:07.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:07.796 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:07.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:07.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:08.061 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:08.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:08.146 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018371 s, 0.0 kB/s 2024-04-26T20:08:08.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:08.246 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:08.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:08.455 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:08.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:08.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:08.672 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272284 s, 0.0 kB/s 2024-04-26T20:08:08.723 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:08.771 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:08.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:08.922 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:09.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:09.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:09.122 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273134 s, 0.0 kB/s 2024-04-26T20:08:09.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:09.295 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:09.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:09.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:09.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:09.674 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:09.748 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224382 s, 0.0 kB/s 2024-04-26T20:08:09.793 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:09.842 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:09.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:09.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:10.108 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:10.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:10.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172808 s, 0.0 kB/s 2024-04-26T20:08:10.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:10.399 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:10.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:10.551 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:10.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:10.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:10.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000134894 s, 0.0 kB/s 2024-04-26T20:08:10.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:10.962 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:11.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:11.149 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:11.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:11.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:11.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00014999 s, 0.0 kB/s 2024-04-26T20:08:11.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:11.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:11.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:11.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:11.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231284 s, 0.0 kB/s 2024-04-26T20:08:12.086 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:12.153 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:12.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:12.318 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:12.385 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:08:12.467 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:08:12.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:12.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:12.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171181 s, 0.0 kB/s 2024-04-26T20:08:12.752 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:12.818 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:12.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:13.035 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:13.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:13.185 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:13.235 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250704 s, 0.0 kB/s 2024-04-26T20:08:13.287 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:13.353 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:13.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:13.510 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:13.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:13.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209425 s, 0.0 kB/s 2024-04-26T20:08:13.837 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:13.894 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:14.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:14.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:14.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:14.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:14.311 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000371462 s, 0.0 kB/s 2024-04-26T20:08:14.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:14.411 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:14.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:14.611 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:14.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:14.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148932 s, 0.0 kB/s 2024-04-26T20:08:14.944 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:14.994 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:15.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:15.170 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:15.319 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:15.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:15.453 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275793 s, 0.0 kB/s 2024-04-26T20:08:15.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:15.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:15.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:15.704 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:15.839 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:15.887 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:15.937 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176718 s, 0.0 kB/s 2024-04-26T20:08:15.989 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:16.037 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:16.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:16.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:16.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:16.432 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152728 s, 0.0 kB/s 2024-04-26T20:08:16.544 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:16.587 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:16.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:16.807 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:17.021 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175105 s, 0.0 kB/s 2024-04-26T20:08:17.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:17.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:17.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:17.413 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:17.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:17.496 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254547 s, 0.0 kB/s 2024-04-26T20:08:17.547 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:17.599 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:17.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:17.821 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:17.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:17.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258937 s, 0.0 kB/s 2024-04-26T20:08:18.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:18.222 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:18.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:18.363 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:18.481 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:18.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:18.563 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015573 s, 0.0 kB/s 2024-04-26T20:08:18.615 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:18.664 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:18.805 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:18.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:18.980 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:19.034 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284756 s, 0.0 kB/s 2024-04-26T20:08:19.115 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:19.164 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:19.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:19.314 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:19.428 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:19.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:19.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181389 s, 0.0 kB/s 2024-04-26T20:08:19.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:19.614 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:19.956 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:20.148 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:20.223 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269517 s, 0.0 kB/s 2024-04-26T20:08:20.294 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:20.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:20.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:20.506 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:20.623 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:20.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:20.734 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188775 s, 0.0 kB/s 2024-04-26T20:08:20.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:20.856 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:20.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:21.031 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:21.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:21.181 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:21.248 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177062 s, 0.0 kB/s 2024-04-26T20:08:21.299 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:21.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:21.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:21.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:21.674 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:21.727 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186802 s, 0.0 kB/s 2024-04-26T20:08:21.783 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:21.832 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:21.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:22.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:22.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:22.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:22.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191023 s, 0.0 kB/s 2024-04-26T20:08:22.275 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:22.324 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:22.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:22.474 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:22.749 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018938 s, 0.0 kB/s 2024-04-26T20:08:22.897 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:22.949 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:23.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:23.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:23.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:23.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:23.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273385 s, 0.0 kB/s 2024-04-26T20:08:23.359 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:23.408 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:23.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:23.566 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:23.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:23.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:23.766 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241382 s, 0.0 kB/s 2024-04-26T20:08:23.817 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:23.866 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:24.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:24.075 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:24.197 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:24.241 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:24.291 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000142277 s, 0.0 kB/s 2024-04-26T20:08:24.343 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:24.392 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:24.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:24.550 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:24.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:24.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171679 s, 0.0 kB/s 2024-04-26T20:08:24.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:24.967 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:25.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:25.159 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:25.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:25.384 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159369 s, 0.0 kB/s 2024-04-26T20:08:25.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:25.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:25.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:25.634 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:25.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:25.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:25.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227187 s, 0.0 kB/s 2024-04-26T20:08:25.948 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:26.025 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:26.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:26.216 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:26.315 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:26.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:26.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199751 s, 0.0 kB/s 2024-04-26T20:08:26.452 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:26.505 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:26.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:26.651 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:26.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:26.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:26.893 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248362 s, 0.0 kB/s 2024-04-26T20:08:26.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:27.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:27.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:27.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:27.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:27.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170666 s, 0.0 kB/s 2024-04-26T20:08:27.503 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:27.568 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:27.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:27.735 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:27.865 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:27.952 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:28.060 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158796 s, 0.0 kB/s 2024-04-26T20:08:28.145 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:28.210 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:28.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:28.377 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:28.509 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:28.644 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205223 s, 0.0 kB/s 2024-04-26T20:08:28.712 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:28.777 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:28.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:28.944 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:29.076 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:29.136 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:29.202 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192883 s, 0.0 kB/s 2024-04-26T20:08:29.277 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:29.336 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:29.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:29.503 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:29.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:29.703 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:29.794 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194876 s, 0.0 kB/s 2024-04-26T20:08:29.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:29.928 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:30.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:30.095 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:30.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:30.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:30.420 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199453 s, 0.0 kB/s 2024-04-26T20:08:30.488 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:30.536 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:30.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:30.691 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:30.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:30.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:30.920 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173099 s, 0.0 kB/s 2024-04-26T20:08:30.979 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:31.037 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:31.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:31.245 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:31.367 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:31.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:31.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151405 s, 0.0 kB/s 2024-04-26T20:08:31.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:31.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:31.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:31.770 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:31.894 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:31.945 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:32.007 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170845 s, 0.0 kB/s 2024-04-26T20:08:32.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:32.120 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:32.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:32.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:32.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:32.454 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:32.512 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268212 s, 0.0 kB/s 2024-04-26T20:08:32.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:32.621 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:32.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:32.771 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:32.885 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:32.921 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:32.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261714 s, 0.0 kB/s 2024-04-26T20:08:33.022 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:33.071 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:33.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:33.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:33.402 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:33.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:33.488 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192697 s, 0.0 kB/s 2024-04-26T20:08:33.539 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:33.588 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:33.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:33.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:33.850 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:33.896 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270291 s, 0.0 kB/s 2024-04-26T20:08:33.998 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:34.047 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:34.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:34.263 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:34.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:34.555 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:34.605 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282124 s, 0.0 kB/s 2024-04-26T20:08:34.693 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:34.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:34.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:34.897 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:35.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000315222 s, 0.0 kB/s 2024-04-26T20:08:35.265 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:35.314 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:35.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:35.481 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:35.595 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:35.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:35.681 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155402 s, 0.0 kB/s 2024-04-26T20:08:35.732 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:35.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:35.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:35.931 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:36.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:36.148 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156127 s, 0.0 kB/s 2024-04-26T20:08:36.199 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:36.248 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:36.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:36.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:36.656 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242205 s, 0.0 kB/s 2024-04-26T20:08:36.716 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:36.756 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:36.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:36.906 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:37.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:37.195 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165016 s, 0.0 kB/s 2024-04-26T20:08:37.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:37.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:37.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:37.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:37.607 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:37.679 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:37.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188138 s, 0.0 kB/s 2024-04-26T20:08:37.908 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:37.957 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:38.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:38.107 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:38.220 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:38.257 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:38.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238103 s, 0.0 kB/s 2024-04-26T20:08:38.359 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:38.407 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:38.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:38.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:38.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:38.828 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227106 s, 0.0 kB/s 2024-04-26T20:08:38.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:38.932 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:39.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:39.133 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:39.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:39.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:39.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020332 s, 0.0 kB/s 2024-04-26T20:08:39.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:39.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:39.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:39.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:39.791 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:39.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002457 s, 0.0 kB/s 2024-04-26T20:08:39.926 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:40.002 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:40.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:40.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:40.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:40.275 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:40.308 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186736 s, 0.0 kB/s 2024-04-26T20:08:40.343 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:40.416 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:40.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:40.567 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:40.669 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:40.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235764 s, 0.0 kB/s 2024-04-26T20:08:40.768 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:40.815 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:40.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:40.934 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:41.037 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:41.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:41.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260468 s, 0.0 kB/s 2024-04-26T20:08:41.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:41.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:41.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:41.443 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:41.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:41.609 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261715 s, 0.0 kB/s 2024-04-26T20:08:41.644 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:41.676 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:41.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:41.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:42.028 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:42.051 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000298038 s, 0.0 kB/s 2024-04-26T20:08:42.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:42.193 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:42.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:42.326 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:42.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:42.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:42.635 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025122 s, 0.0 kB/s 2024-04-26T20:08:42.670 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:42.702 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:42.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:42.860 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:42.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:42.994 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:43.027 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155296 s, 0.0 kB/s 2024-04-26T20:08:43.087 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:43.127 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:43.161 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:08:43.233 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:08:43.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:43.310 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:43.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:43.544 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169367 s, 0.0 kB/s 2024-04-26T20:08:43.579 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:43.611 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:43.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:43.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:43.884 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:43.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:44.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024064 s, 0.0 kB/s 2024-04-26T20:08:44.046 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:44.104 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:44.238 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:44.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:44.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:44.420 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179682 s, 0.0 kB/s 2024-04-26T20:08:44.454 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:44.486 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:44.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:44.645 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159873 s, 0.0 kB/s 2024-04-26T20:08:44.874 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:44.903 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:44.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:45.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:45.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:45.170 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:45.204 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199929 s, 0.0 kB/s 2024-04-26T20:08:45.289 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:45.329 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:45.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:45.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:45.601 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:45.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:45.662 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186215 s, 0.0 kB/s 2024-04-26T20:08:45.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:45.729 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:45.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:45.863 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:45.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:45.996 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:46.089 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192274 s, 0.0 kB/s 2024-04-26T20:08:46.147 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:46.217 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:46.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:46.346 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:46.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:46.488 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:46.525 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161987 s, 0.0 kB/s 2024-04-26T20:08:46.556 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:46.588 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:46.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:46.697 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:46.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:46.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:46.904 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017323 s, 0.0 kB/s 2024-04-26T20:08:46.940 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:46.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:47.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:47.095 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:47.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:47.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:47.340 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236677 s, 0.0 kB/s 2024-04-26T20:08:47.373 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:47.406 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:47.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:47.614 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:47.711 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:47.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:47.786 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196002 s, 0.0 kB/s 2024-04-26T20:08:47.864 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:47.889 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:47.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:48.054 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:48.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:48.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:48.231 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017735 s, 0.0 kB/s 2024-04-26T20:08:48.266 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:48.298 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:48.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:48.473 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:48.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188616 s, 0.0 kB/s 2024-04-26T20:08:48.649 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:48.714 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:48.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:48.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:48.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:49.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:49.040 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256742 s, 0.0 kB/s 2024-04-26T20:08:49.066 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:49.099 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:49.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:49.224 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:49.316 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:49.363 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:49.391 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025264 s, 0.0 kB/s 2024-04-26T20:08:49.417 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:49.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:49.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:49.633 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:49.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:49.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:49.799 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242753 s, 0.0 kB/s 2024-04-26T20:08:49.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:49.866 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:49.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:49.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:50.166 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000131299 s, 0.0 kB/s 2024-04-26T20:08:50.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:50.233 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:50.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:50.383 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:50.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:50.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:50.541 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246958 s, 0.0 kB/s 2024-04-26T20:08:50.576 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:50.608 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:50.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:50.736 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:50.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:50.892 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018839 s, 0.0 kB/s 2024-04-26T20:08:50.926 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:50.984 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:51.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:51.150 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:51.275 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:51.309 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179489 s, 0.0 kB/s 2024-04-26T20:08:51.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:51.475 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:51.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:51.640 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:51.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:51.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224556 s, 0.0 kB/s 2024-04-26T20:08:51.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:51.876 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:51.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:51.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:52.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:52.292 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268189 s, 0.0 kB/s 2024-04-26T20:08:52.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:52.410 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:52.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:52.543 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:52.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:52.701 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163306 s, 0.0 kB/s 2024-04-26T20:08:52.736 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:52.768 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:52.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:52.902 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:53.002 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197673 s, 0.0 kB/s 2024-04-26T20:08:53.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:53.210 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:53.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:53.410 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:53.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:53.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:53.600 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224781 s, 0.0 kB/s 2024-04-26T20:08:53.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:53.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:53.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:53.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:53.892 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:54.002 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153857 s, 0.0 kB/s 2024-04-26T20:08:54.037 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:54.069 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:54.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:54.199 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:54.299 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:54.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:54.361 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002127 s, 0.0 kB/s 2024-04-26T20:08:54.395 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:54.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:54.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:54.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:54.759 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:54.794 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:54.828 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220378 s, 0.0 kB/s 2024-04-26T20:08:54.862 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:54.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:54.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:55.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:55.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:55.153 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:55.260 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026292 s, 0.0 kB/s 2024-04-26T20:08:55.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:55.341 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:55.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:55.495 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:55.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:55.662 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:55.729 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211242 s, 0.0 kB/s 2024-04-26T20:08:55.763 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:55.837 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:55.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:55.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:56.054 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:56.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:56.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235627 s, 0.0 kB/s 2024-04-26T20:08:56.197 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:56.229 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:56.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:56.448 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:56.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:56.629 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:56.663 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196846 s, 0.0 kB/s 2024-04-26T20:08:56.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:56.759 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:56.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:56.930 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:57.029 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:57.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:57.122 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163912 s, 0.0 kB/s 2024-04-26T20:08:57.155 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:57.188 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:57.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:57.305 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:57.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:57.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:57.530 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230941 s, 0.0 kB/s 2024-04-26T20:08:57.582 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:57.630 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:57.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:57.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:57.896 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:57.925 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:57.982 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257258 s, 0.0 kB/s 2024-04-26T20:08:58.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:58.047 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:58.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:58.164 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:58.297 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:58.331 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016044 s, 0.0 kB/s 2024-04-26T20:08:58.407 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:58.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:58.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:58.573 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:58.726 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:58.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:58.798 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000134158 s, 0.0 kB/s 2024-04-26T20:08:58.832 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:58.898 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:58.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:59.015 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:59.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:59.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243607 s, 0.0 kB/s 2024-04-26T20:08:59.299 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:59.331 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:59.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:59.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:08:59.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:08:59.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:08:59.717 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175033 s, 0.0 kB/s 2024-04-26T20:08:59.750 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:08:59.799 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:08:59.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:08:59.940 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:00.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:00.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:00.174 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265872 s, 0.0 kB/s 2024-04-26T20:09:00.209 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:00.241 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:00.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:00.360 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:00.460 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:00.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292579 s, 0.0 kB/s 2024-04-26T20:09:00.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:00.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:00.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:00.811 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:00.909 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:00.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180192 s, 0.0 kB/s 2024-04-26T20:09:01.004 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:01.036 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:01.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:01.169 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:01.271 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:01.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:01.336 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177961 s, 0.0 kB/s 2024-04-26T20:09:01.371 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:01.403 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:01.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:01.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:01.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:01.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:01.778 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195997 s, 0.0 kB/s 2024-04-26T20:09:01.813 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:01.845 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:01.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:01.986 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:02.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:02.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214953 s, 0.0 kB/s 2024-04-26T20:09:02.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:02.253 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:02.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:02.453 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:02.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:02.603 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:02.653 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019762 s, 0.0 kB/s 2024-04-26T20:09:02.705 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:02.754 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:02.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:02.929 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:03.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:03.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213295 s, 0.0 kB/s 2024-04-26T20:09:03.188 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:03.237 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:03.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:03.379 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:03.654 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246208 s, 0.0 kB/s 2024-04-26T20:09:03.705 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:03.808 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:03.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:03.988 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:04.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230437 s, 0.0 kB/s 2024-04-26T20:09:04.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:04.337 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:04.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:04.488 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:04.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:04.713 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015674 s, 0.0 kB/s 2024-04-26T20:09:04.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:04.830 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:04.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:04.963 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:05.089 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:05.184 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258196 s, 0.0 kB/s 2024-04-26T20:09:05.231 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:05.305 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:05.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:05.472 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:05.588 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:05.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:05.688 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226492 s, 0.0 kB/s 2024-04-26T20:09:05.740 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:05.805 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:05.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:06.030 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:06.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:06.180 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:06.231 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215523 s, 0.0 kB/s 2024-04-26T20:09:06.281 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:06.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:06.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:06.497 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:06.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:06.714 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:06.764 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213372 s, 0.0 kB/s 2024-04-26T20:09:06.815 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:06.864 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:06.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:07.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:07.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:07.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:07.214 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018738 s, 0.0 kB/s 2024-04-26T20:09:07.265 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:07.351 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:07.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:07.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:07.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:07.756 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154746 s, 0.0 kB/s 2024-04-26T20:09:07.807 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:07.856 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:08.006 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:08.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:08.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:08.206 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193303 s, 0.0 kB/s 2024-04-26T20:09:08.258 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:08.306 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:08.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:08.473 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:08.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:08.698 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183256 s, 0.0 kB/s 2024-04-26T20:09:08.749 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:08.801 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:08.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:08.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:09.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:09.099 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:09.149 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207482 s, 0.0 kB/s 2024-04-26T20:09:09.225 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:09.292 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:09.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:09.457 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:09.607 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:09.774 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249283 s, 0.0 kB/s 2024-04-26T20:09:09.825 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:09.894 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:10.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:10.050 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:10.176 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:10.274 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:10.324 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212977 s, 0.0 kB/s 2024-04-26T20:09:10.375 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:10.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:10.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:10.625 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:10.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:10.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:10.850 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177614 s, 0.0 kB/s 2024-04-26T20:09:10.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:10.950 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:11.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:11.150 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:11.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:11.300 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:11.350 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242667 s, 0.0 kB/s 2024-04-26T20:09:11.401 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:11.453 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:11.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:11.600 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:11.717 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:11.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:11.801 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179653 s, 0.0 kB/s 2024-04-26T20:09:11.852 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:11.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196866 s, 0.0 kB/s 2024-04-26T20:09:12.193 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.193 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.244 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.307 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166698 s, 0.0 kB/s 2024-04-26T20:09:12.308 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.358 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.463 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170626 s, 0.0 kB/s 2024-04-26T20:09:12.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.464 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.517 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228762 s, 0.0 kB/s 2024-04-26T20:09:12.581 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.581 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.697 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219137 s, 0.0 kB/s 2024-04-26T20:09:12.699 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.699 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.810 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167485 s, 0.0 kB/s 2024-04-26T20:09:12.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.811 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.866 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:12.925 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:12.926 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:12.926 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015811 s, 0.0 kB/s 2024-04-26T20:09:12.927 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:12.927 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:12.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:12.976 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.081 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.081 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189882 s, 0.0 kB/s 2024-04-26T20:09:13.082 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.134 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.191 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000326385 s, 0.0 kB/s 2024-04-26T20:09:13.192 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.193 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.246 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.303 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284125 s, 0.0 kB/s 2024-04-26T20:09:13.304 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.304 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.357 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.416 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.416 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.416 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165852 s, 0.0 kB/s 2024-04-26T20:09:13.417 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.417 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.574 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231583 s, 0.0 kB/s 2024-04-26T20:09:13.575 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.575 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.579 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:09:13.585 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:09:13.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.621 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.722 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.722 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.722 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164743 s, 0.0 kB/s 2024-04-26T20:09:13.722 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.722 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.774 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246747 s, 0.0 kB/s 2024-04-26T20:09:13.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.834 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:13.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:13.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:13.940 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169551 s, 0.0 kB/s 2024-04-26T20:09:13.941 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:13.941 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:13.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:13.990 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.045 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221963 s, 0.0 kB/s 2024-04-26T20:09:14.046 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.047 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.099 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.158 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021771 s, 0.0 kB/s 2024-04-26T20:09:14.159 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.159 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.213 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.272 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157609 s, 0.0 kB/s 2024-04-26T20:09:14.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.273 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.325 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.384 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.384 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.384 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000141044 s, 0.0 kB/s 2024-04-26T20:09:14.385 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.436 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.496 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260798 s, 0.0 kB/s 2024-04-26T20:09:14.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.497 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.548 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.653 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.653 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.653 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233791 s, 0.0 kB/s 2024-04-26T20:09:14.654 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.654 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.704 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.808 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228313 s, 0.0 kB/s 2024-04-26T20:09:14.809 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.809 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.861 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:14.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:14.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:14.924 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199782 s, 0.0 kB/s 2024-04-26T20:09:14.925 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:14.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:14.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:14.978 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.039 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212853 s, 0.0 kB/s 2024-04-26T20:09:15.040 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.040 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.092 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.152 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.152 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.152 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179095 s, 0.0 kB/s 2024-04-26T20:09:15.153 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.153 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.205 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018702 s, 0.0 kB/s 2024-04-26T20:09:15.310 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.366 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.424 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224179 s, 0.0 kB/s 2024-04-26T20:09:15.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.477 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.583 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286818 s, 0.0 kB/s 2024-04-26T20:09:15.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.703 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164166 s, 0.0 kB/s 2024-04-26T20:09:15.704 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.704 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.756 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.807 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195291 s, 0.0 kB/s 2024-04-26T20:09:15.808 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.809 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:15.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:15.858 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:15.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:15.965 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:15.965 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217654 s, 0.0 kB/s 2024-04-26T20:09:15.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:15.967 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:16.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:16.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:16.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:16.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156822 s, 0.0 kB/s 2024-04-26T20:09:16.083 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:16.083 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:16.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:16.209 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:16.332 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:16.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:16.418 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218146 s, 0.0 kB/s 2024-04-26T20:09:16.469 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:16.551 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:16.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:16.701 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:16.851 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242051 s, 0.0 kB/s 2024-04-26T20:09:16.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:16.985 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:17.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:17.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:17.253 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224249 s, 0.0 kB/s 2024-04-26T20:09:17.366 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:17.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:17.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:17.535 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:17.642 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:17.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:17.744 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244041 s, 0.0 kB/s 2024-04-26T20:09:17.786 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:17.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:17.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:17.952 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:18.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:18.163 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023976 s, 0.0 kB/s 2024-04-26T20:09:18.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:18.312 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:18.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:18.419 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:18.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:18.561 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:18.594 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018449 s, 0.0 kB/s 2024-04-26T20:09:18.629 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:18.661 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:18.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:18.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:18.883 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:18.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:18.953 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210239 s, 0.0 kB/s 2024-04-26T20:09:18.987 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:19.020 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:19.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:19.145 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:19.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:19.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:19.328 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204097 s, 0.0 kB/s 2024-04-26T20:09:19.363 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:19.395 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:19.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:19.645 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209236 s, 0.0 kB/s 2024-04-26T20:09:19.863 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:19.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:20.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:20.029 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:20.170 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:20.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:20.280 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204755 s, 0.0 kB/s 2024-04-26T20:09:20.313 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:20.379 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:20.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:20.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:20.594 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:20.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:20.654 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198062 s, 0.0 kB/s 2024-04-26T20:09:20.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:20.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:20.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:20.929 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:21.045 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:21.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:21.163 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189211 s, 0.0 kB/s 2024-04-26T20:09:21.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:21.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:21.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:21.438 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:21.551 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:21.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:21.663 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224462 s, 0.0 kB/s 2024-04-26T20:09:21.715 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:21.778 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:21.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:21.947 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:22.065 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:22.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:22.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022588 s, 0.0 kB/s 2024-04-26T20:09:22.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:22.272 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:22.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:22.422 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:22.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:22.695 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173456 s, 0.0 kB/s 2024-04-26T20:09:22.773 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:22.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:22.981 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:23.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:23.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:23.181 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190805 s, 0.0 kB/s 2024-04-26T20:09:23.232 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:23.281 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:23.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:23.439 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:23.576 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:23.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194672 s, 0.0 kB/s 2024-04-26T20:09:23.757 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:23.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:23.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:23.998 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:24.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:24.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196959 s, 0.0 kB/s 2024-04-26T20:09:24.300 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:24.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:24.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:24.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:24.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:24.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:24.715 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200449 s, 0.0 kB/s 2024-04-26T20:09:24.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:24.824 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:24.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:24.999 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:25.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252934 s, 0.0 kB/s 2024-04-26T20:09:25.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:25.382 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:25.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:25.557 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:25.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:25.783 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:25.833 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266128 s, 0.0 kB/s 2024-04-26T20:09:25.884 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:25.933 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:26.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:26.083 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:26.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:26.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:26.308 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264046 s, 0.0 kB/s 2024-04-26T20:09:26.389 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:26.483 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:26.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:26.658 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:26.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:26.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:26.867 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149588 s, 0.0 kB/s 2024-04-26T20:09:26.918 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:26.968 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:27.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:27.117 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:27.233 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:27.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:27.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194131 s, 0.0 kB/s 2024-04-26T20:09:27.368 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:27.442 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:27.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:27.592 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:27.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256909 s, 0.0 kB/s 2024-04-26T20:09:27.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:27.917 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:28.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:28.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:28.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:28.234 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:28.284 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021781 s, 0.0 kB/s 2024-04-26T20:09:28.386 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:28.434 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:28.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:28.584 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292234 s, 0.0 kB/s 2024-04-26T20:09:28.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:28.884 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:29.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:29.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:29.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:29.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:29.360 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178912 s, 0.0 kB/s 2024-04-26T20:09:29.411 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:29.464 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:29.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:29.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:29.756 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:29.806 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:29.864 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189699 s, 0.0 kB/s 2024-04-26T20:09:29.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:30.076 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:30.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:30.264 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:30.382 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:30.440 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:30.515 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017409 s, 0.0 kB/s 2024-04-26T20:09:30.566 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:30.633 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:30.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:30.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:30.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:30.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:30.998 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180987 s, 0.0 kB/s 2024-04-26T20:09:31.049 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:31.098 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:31.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:31.298 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:31.416 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:31.457 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:31.507 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236861 s, 0.0 kB/s 2024-04-26T20:09:31.580 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:31.632 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:31.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:31.774 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:31.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:31.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:32.030 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192014 s, 0.0 kB/s 2024-04-26T20:09:32.075 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:32.124 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:32.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:32.316 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:32.433 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:32.549 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222516 s, 0.0 kB/s 2024-04-26T20:09:32.648 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:32.682 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:32.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:32.841 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:32.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:33.049 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018317 s, 0.0 kB/s 2024-04-26T20:09:33.101 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:33.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:33.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:33.350 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:33.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:33.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:33.641 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266979 s, 0.0 kB/s 2024-04-26T20:09:33.692 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:33.741 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:33.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:33.925 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:34.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153524 s, 0.0 kB/s 2024-04-26T20:09:34.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:34.300 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:34.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:34.450 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:34.642 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:34.701 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227316 s, 0.0 kB/s 2024-04-26T20:09:34.752 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:34.801 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:34.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:34.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:35.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:35.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:35.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193347 s, 0.0 kB/s 2024-04-26T20:09:35.244 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:35.317 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:35.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:35.468 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:35.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:35.693 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192468 s, 0.0 kB/s 2024-04-26T20:09:35.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:35.793 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:35.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:36.009 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:36.125 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:36.160 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:36.210 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205299 s, 0.0 kB/s 2024-04-26T20:09:36.263 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:36.310 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:36.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:36.485 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:36.641 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:36.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207223 s, 0.0 kB/s 2024-04-26T20:09:36.786 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:36.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:37.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:37.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:37.192 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:37.277 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222791 s, 0.0 kB/s 2024-04-26T20:09:37.328 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:37.377 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:37.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:37.527 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:37.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:37.711 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:37.761 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181936 s, 0.0 kB/s 2024-04-26T20:09:37.846 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:37.894 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:38.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:38.044 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:38.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:38.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:38.274 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158985 s, 0.0 kB/s 2024-04-26T20:09:38.321 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:38.400 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:38.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:38.544 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:38.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:38.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:38.844 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192298 s, 0.0 kB/s 2024-04-26T20:09:38.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:38.956 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:39.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:39.112 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:39.227 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:39.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:39.312 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182967 s, 0.0 kB/s 2024-04-26T20:09:39.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:39.412 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:39.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:39.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:39.756 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:39.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:39.862 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195227 s, 0.0 kB/s 2024-04-26T20:09:39.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:39.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:40.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:40.146 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:40.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000375786 s, 0.0 kB/s 2024-04-26T20:09:40.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:40.514 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:40.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:40.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:40.862 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:40.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:40.996 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201696 s, 0.0 kB/s 2024-04-26T20:09:41.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:41.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:41.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:41.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:41.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:41.563 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211553 s, 0.0 kB/s 2024-04-26T20:09:41.687 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:41.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:41.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:41.913 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:42.044 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:42.146 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:42.213 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188938 s, 0.0 kB/s 2024-04-26T20:09:42.281 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:42.346 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:42.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:42.513 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:42.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:42.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:42.822 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229081 s, 0.0 kB/s 2024-04-26T20:09:42.890 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:42.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:43.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:43.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:43.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:43.439 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189035 s, 0.0 kB/s 2024-04-26T20:09:43.507 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:43.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:43.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:43.731 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:43.782 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:09:43.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:09:43.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:44.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:44.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179063 s, 0.0 kB/s 2024-04-26T20:09:44.139 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:44.197 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:44.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:44.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:44.552 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:44.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:44.656 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182142 s, 0.0 kB/s 2024-04-26T20:09:44.715 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:44.764 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:44.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:44.956 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:45.099 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:45.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286247 s, 0.0 kB/s 2024-04-26T20:09:45.241 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:45.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:45.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:45.431 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:45.547 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:45.682 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:45.770 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199258 s, 0.0 kB/s 2024-04-26T20:09:45.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:45.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:46.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:46.132 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:46.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:46.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176755 s, 0.0 kB/s 2024-04-26T20:09:46.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:46.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:46.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:46.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:46.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:46.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:46.890 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214613 s, 0.0 kB/s 2024-04-26T20:09:46.942 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:46.991 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:47.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:47.141 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:47.258 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:47.291 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:47.341 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193573 s, 0.0 kB/s 2024-04-26T20:09:47.392 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:47.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:47.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:47.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:47.707 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:47.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:47.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190461 s, 0.0 kB/s 2024-04-26T20:09:47.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:47.942 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:48.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:48.092 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:48.211 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:48.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:48.353 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272476 s, 0.0 kB/s 2024-04-26T20:09:48.401 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:48.450 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:48.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:48.609 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:48.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:48.784 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:48.834 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193466 s, 0.0 kB/s 2024-04-26T20:09:48.885 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:48.936 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:49.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:49.084 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:49.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:49.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:49.317 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250226 s, 0.0 kB/s 2024-04-26T20:09:49.369 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:49.426 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:49.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:49.584 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:49.762 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:49.801 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:49.851 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000139767 s, 0.0 kB/s 2024-04-26T20:09:49.903 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:49.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:50.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:50.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:50.243 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:50.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:50.339 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243608 s, 0.0 kB/s 2024-04-26T20:09:50.377 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:50.426 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:50.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:50.610 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:50.858 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:51.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:51.102 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149249 s, 0.0 kB/s 2024-04-26T20:09:51.153 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:51.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:51.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:51.352 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:51.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:51.569 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244338 s, 0.0 kB/s 2024-04-26T20:09:51.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:51.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:51.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:51.844 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:52.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:52.139 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263598 s, 0.0 kB/s 2024-04-26T20:09:52.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:52.219 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:52.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:52.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:52.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:52.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:52.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000289601 s, 0.0 kB/s 2024-04-26T20:09:52.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:52.711 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:52.878 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:52.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:53.063 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:53.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200181 s, 0.0 kB/s 2024-04-26T20:09:53.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:53.211 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:53.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:53.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:53.477 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:53.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251491 s, 0.0 kB/s 2024-04-26T20:09:53.679 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:53.728 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:53.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:53.878 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:54.028 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:54.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161162 s, 0.0 kB/s 2024-04-26T20:09:54.130 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:54.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:54.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:54.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:54.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:54.604 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000293431 s, 0.0 kB/s 2024-04-26T20:09:54.655 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:54.704 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:54.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:54.854 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:54.970 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:55.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:55.054 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231532 s, 0.0 kB/s 2024-04-26T20:09:55.105 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:55.162 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:55.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:55.321 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:55.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:55.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:55.554 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192322 s, 0.0 kB/s 2024-04-26T20:09:55.606 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:55.684 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:55.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:56.069 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:56.213 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:56.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:56.297 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215708 s, 0.0 kB/s 2024-04-26T20:09:56.348 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:56.396 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:56.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:56.547 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:56.662 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:56.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:56.763 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261431 s, 0.0 kB/s 2024-04-26T20:09:56.832 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:56.872 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:56.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:57.022 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:57.171 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:57.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:57.272 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159555 s, 0.0 kB/s 2024-04-26T20:09:57.324 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:57.397 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:57.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:57.547 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:57.720 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:57.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:57.823 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213283 s, 0.0 kB/s 2024-04-26T20:09:57.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:57.923 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:58.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:58.111 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:58.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:58.323 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230749 s, 0.0 kB/s 2024-04-26T20:09:58.424 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:58.484 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:58.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:58.623 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:58.740 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:58.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:58.848 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172867 s, 0.0 kB/s 2024-04-26T20:09:58.900 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:58.958 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:59.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:59.141 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:59.254 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:59.323 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:59.406 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219001 s, 0.0 kB/s 2024-04-26T20:09:59.450 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:09:59.539 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:09:59.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:09:59.726 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:09:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:09:59.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:09:59.934 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236949 s, 0.0 kB/s 2024-04-26T20:10:00.055 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:00.107 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:00.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:00.307 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:00.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:00.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:00.574 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232427 s, 0.0 kB/s 2024-04-26T20:10:00.642 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:00.707 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:00.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:00.941 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:01.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:01.271 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:01.333 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257876 s, 0.0 kB/s 2024-04-26T20:10:01.401 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:01.466 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:01.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:01.641 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:01.774 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:01.841 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:01.908 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246624 s, 0.0 kB/s 2024-04-26T20:10:01.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:02.041 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:02.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:02.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:02.466 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015261 s, 0.0 kB/s 2024-04-26T20:10:02.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:02.666 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:02.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:02.921 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:03.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:03.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243017 s, 0.0 kB/s 2024-04-26T20:10:03.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:03.317 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:03.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:03.550 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:03.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279938 s, 0.0 kB/s 2024-04-26T20:10:03.885 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:03.978 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:04.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:04.134 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:04.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:04.324 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:04.367 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222093 s, 0.0 kB/s 2024-04-26T20:10:04.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:04.534 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:04.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:04.701 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:04.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:04.851 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000299001 s, 0.0 kB/s 2024-04-26T20:10:04.952 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:05.161 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:05.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:05.309 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:05.516 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:05.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:05.718 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240112 s, 0.0 kB/s 2024-04-26T20:10:05.785 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:05.907 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:06.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:06.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:06.238 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:06.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261221 s, 0.0 kB/s 2024-04-26T20:10:06.403 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:06.452 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:06.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:06.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:06.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:06.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:06.877 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181098 s, 0.0 kB/s 2024-04-26T20:10:06.953 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:07.002 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:07.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:07.152 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:07.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:07.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:07.378 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198835 s, 0.0 kB/s 2024-04-26T20:10:07.429 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:07.478 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:07.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:07.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:07.768 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:07.828 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:07.878 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000149401 s, 0.0 kB/s 2024-04-26T20:10:07.953 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:08.003 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:08.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:08.153 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:08.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180445 s, 0.0 kB/s 2024-04-26T20:10:08.472 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:08.520 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:08.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:08.682 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:08.817 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:08.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:08.928 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154381 s, 0.0 kB/s 2024-04-26T20:10:08.981 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:09.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:09.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:09.179 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:09.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:09.387 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:09.437 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000240536 s, 0.0 kB/s 2024-04-26T20:10:09.489 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:09.550 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:09.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:09.729 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:09.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:09.862 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:09.912 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177201 s, 0.0 kB/s 2024-04-26T20:10:09.964 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:10.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:10.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:10.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:10.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:10.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:10.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236276 s, 0.0 kB/s 2024-04-26T20:10:10.456 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:10.504 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:10.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:10.688 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:10.816 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:10.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:10.945 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262262 s, 0.0 kB/s 2024-04-26T20:10:10.981 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:11.063 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:11.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:11.247 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:11.363 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:11.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:11.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265181 s, 0.0 kB/s 2024-04-26T20:10:11.498 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:11.546 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:11.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:11.705 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:11.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:11.863 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:11.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202994 s, 0.0 kB/s 2024-04-26T20:10:11.965 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:12.013 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:12.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:12.164 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:12.278 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172006 s, 0.0 kB/s 2024-04-26T20:10:12.450 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:12.531 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:12.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:12.681 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:12.797 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:12.881 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000282467 s, 0.0 kB/s 2024-04-26T20:10:12.932 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:13.047 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:13.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:13.214 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:13.331 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:13.412 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:13.464 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230257 s, 0.0 kB/s 2024-04-26T20:10:13.516 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:13.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:13.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:13.748 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:13.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:13.988 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:14.040 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220593 s, 0.0 kB/s 2024-04-26T20:10:14.091 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:14.160 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:14.260 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:10:14.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:14.348 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:14.399 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:10:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:14.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:14.623 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175863 s, 0.0 kB/s 2024-04-26T20:10:14.675 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:14.723 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:14.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:14.932 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:15.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:15.085 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:15.132 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166255 s, 0.0 kB/s 2024-04-26T20:10:15.183 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:15.271 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:15.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:15.407 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:15.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:15.582 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:15.632 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220097 s, 0.0 kB/s 2024-04-26T20:10:15.692 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:15.774 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:15.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:15.924 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:16.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:16.124 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258548 s, 0.0 kB/s 2024-04-26T20:10:16.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:16.224 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:16.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:16.408 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:16.524 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:16.558 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:16.682 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175093 s, 0.0 kB/s 2024-04-26T20:10:16.726 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:16.797 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:16.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:17.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:17.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:17.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:17.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246079 s, 0.0 kB/s 2024-04-26T20:10:17.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:17.326 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:17.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:17.475 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:17.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:17.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:17.715 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183201 s, 0.0 kB/s 2024-04-26T20:10:17.760 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:17.825 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:17.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:17.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:18.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:18.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206012 s, 0.0 kB/s 2024-04-26T20:10:18.261 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:18.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:18.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:18.557 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:18.668 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:18.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000139883 s, 0.0 kB/s 2024-04-26T20:10:18.802 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:18.851 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:19.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:19.043 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:19.196 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:19.260 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:19.310 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184956 s, 0.0 kB/s 2024-04-26T20:10:19.375 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:19.410 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:19.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:19.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:19.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:19.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:19.793 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155298 s, 0.0 kB/s 2024-04-26T20:10:19.845 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:19.918 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:20.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:20.077 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:20.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:20.277 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:20.366 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221365 s, 0.0 kB/s 2024-04-26T20:10:20.453 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:20.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:20.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:20.827 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:20.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:21.094 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221001 s, 0.0 kB/s 2024-04-26T20:10:21.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:21.227 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:21.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:21.394 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:21.527 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:21.657 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:21.719 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000309627 s, 0.0 kB/s 2024-04-26T20:10:21.787 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:21.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:21.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:22.019 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:22.228 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197031 s, 0.0 kB/s 2024-04-26T20:10:22.362 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:22.420 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:22.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:22.620 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:22.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:22.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:22.870 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015985 s, 0.0 kB/s 2024-04-26T20:10:22.921 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:22.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:23.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:23.128 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:23.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:23.278 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:23.345 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220983 s, 0.0 kB/s 2024-04-26T20:10:23.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:23.469 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:23.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:23.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:23.755 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:23.895 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245077 s, 0.0 kB/s 2024-04-26T20:10:23.955 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:24.012 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:24.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:24.162 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:24.278 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:24.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023998 s, 0.0 kB/s 2024-04-26T20:10:24.413 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:24.465 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:24.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:24.679 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:24.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:24.879 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000266152 s, 0.0 kB/s 2024-04-26T20:10:24.987 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:25.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:25.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:25.179 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:25.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:25.330 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:25.380 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186491 s, 0.0 kB/s 2024-04-26T20:10:25.431 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:25.530 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:25.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:25.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:25.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:25.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:25.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000236649 s, 0.0 kB/s 2024-04-26T20:10:25.956 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:26.005 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:26.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:26.155 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:26.270 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:26.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:26.443 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026477 s, 0.0 kB/s 2024-04-26T20:10:26.498 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:26.547 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:26.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:26.697 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:26.815 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:26.914 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264858 s, 0.0 kB/s 2024-04-26T20:10:26.957 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:27.005 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:27.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:27.239 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:27.355 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:27.389 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:27.439 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224173 s, 0.0 kB/s 2024-04-26T20:10:27.511 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:27.564 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:27.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:27.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:27.839 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:27.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:27.948 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021994 s, 0.0 kB/s 2024-04-26T20:10:27.999 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:28.073 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:28.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:28.250 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:28.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:28.392 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:28.425 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172254 s, 0.0 kB/s 2024-04-26T20:10:28.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:28.492 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:28.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:28.626 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:28.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:28.884 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166701 s, 0.0 kB/s 2024-04-26T20:10:28.918 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:28.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:29.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:29.085 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:29.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:29.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:29.251 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161237 s, 0.0 kB/s 2024-04-26T20:10:29.285 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:29.347 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:29.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:29.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:29.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:29.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:29.710 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241453 s, 0.0 kB/s 2024-04-26T20:10:29.744 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:29.776 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:29.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:29.893 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:29.992 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167418 s, 0.0 kB/s 2024-04-26T20:10:30.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:30.171 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:30.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:30.310 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:30.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:30.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:30.544 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168926 s, 0.0 kB/s 2024-04-26T20:10:30.578 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:30.658 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:30.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:30.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:30.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:30.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:31.002 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000312803 s, 0.0 kB/s 2024-04-26T20:10:31.037 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:31.069 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:31.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:31.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:31.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:31.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:31.444 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166832 s, 0.0 kB/s 2024-04-26T20:10:31.491 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:31.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:31.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:31.655 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:31.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:31.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239147 s, 0.0 kB/s 2024-04-26T20:10:31.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:31.920 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:32.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:32.047 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:32.209 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:32.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:32.328 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246934 s, 0.0 kB/s 2024-04-26T20:10:32.414 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:32.445 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:32.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:32.562 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:32.651 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:32.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:32.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020028 s, 0.0 kB/s 2024-04-26T20:10:32.825 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:32.845 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:32.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:32.987 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:33.077 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:33.141 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:33.179 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016135 s, 0.0 kB/s 2024-04-26T20:10:33.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:33.262 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:33.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:33.440 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:33.545 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:33.600 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:33.677 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153911 s, 0.0 kB/s 2024-04-26T20:10:33.714 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:33.754 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:33.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:33.937 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:34.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:34.063 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:34.104 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162867 s, 0.0 kB/s 2024-04-26T20:10:34.139 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:34.171 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:34.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:34.355 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:34.479 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:34.549 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185462 s, 0.0 kB/s 2024-04-26T20:10:34.606 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:34.646 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:34.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:34.805 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:34.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:35.015 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211327 s, 0.0 kB/s 2024-04-26T20:10:35.067 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:35.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:35.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:35.213 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:35.347 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:35.381 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279144 s, 0.0 kB/s 2024-04-26T20:10:35.422 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:35.531 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:35.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:35.706 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:35.922 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:35.948 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022042 s, 0.0 kB/s 2024-04-26T20:10:35.990 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:36.023 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:36.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:36.198 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:36.296 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:36.383 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197612 s, 0.0 kB/s 2024-04-26T20:10:36.424 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:36.498 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:36.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:36.615 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:36.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:36.840 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177776 s, 0.0 kB/s 2024-04-26T20:10:36.875 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:36.981 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:37.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:37.124 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:37.294 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:37.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:37.382 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020363 s, 0.0 kB/s 2024-04-26T20:10:37.416 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:37.449 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:37.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:37.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:37.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:37.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:37.832 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001974 s, 0.0 kB/s 2024-04-26T20:10:37.867 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:37.919 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:38.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:38.057 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:38.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:38.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227942 s, 0.0 kB/s 2024-04-26T20:10:38.259 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:38.342 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:38.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:38.458 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:38.560 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:38.591 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:38.641 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174794 s, 0.0 kB/s 2024-04-26T20:10:38.676 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:38.708 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:38.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:38.833 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:38.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:39.025 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226009 s, 0.0 kB/s 2024-04-26T20:10:39.151 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:39.208 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:39.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:39.334 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:39.442 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:39.587 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:39.617 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186658 s, 0.0 kB/s 2024-04-26T20:10:39.670 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:39.700 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:39.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:39.817 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:39.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:39.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000249867 s, 0.0 kB/s 2024-04-26T20:10:40.045 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:40.084 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:40.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:40.226 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:40.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:40.359 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:40.418 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241352 s, 0.0 kB/s 2024-04-26T20:10:40.501 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:40.536 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:40.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:40.718 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:40.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:40.909 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020623 s, 0.0 kB/s 2024-04-26T20:10:40.944 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:41.001 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:41.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:41.118 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:41.249 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:41.405 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186002 s, 0.0 kB/s 2024-04-26T20:10:41.463 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:41.527 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:41.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:41.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:41.751 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:41.785 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:41.818 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156993 s, 0.0 kB/s 2024-04-26T20:10:41.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:41.905 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:42.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:42.035 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:42.136 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:42.219 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252164 s, 0.0 kB/s 2024-04-26T20:10:42.337 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:42.369 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:42.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:42.538 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:42.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:42.669 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:42.781 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150306 s, 0.0 kB/s 2024-04-26T20:10:42.821 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:42.853 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:42.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:43.019 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:43.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:43.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:43.178 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193653 s, 0.0 kB/s 2024-04-26T20:10:43.212 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:43.245 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:43.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:43.362 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:43.520 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:43.553 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171977 s, 0.0 kB/s 2024-04-26T20:10:43.613 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:43.645 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:43.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:43.828 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:43.979 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:44.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181821 s, 0.0 kB/s 2024-04-26T20:10:44.047 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:44.079 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:44.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:44.220 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:44.316 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:44.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:44.404 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000399216 s, 0.0 kB/s 2024-04-26T20:10:44.439 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:44.509 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:44.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:44.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:44.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:10:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:44.879 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164557 s, 0.0 kB/s 2024-04-26T20:10:44.996 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:45.030 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:45.064 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:10:45.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:45.180 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:45.279 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:45.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:45.376 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227357 s, 0.0 kB/s 2024-04-26T20:10:45.415 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:45.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:45.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:45.619 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:45.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:45.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203643 s, 0.0 kB/s 2024-04-26T20:10:45.946 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:45.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:46.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:46.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:46.280 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:46.314 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199492 s, 0.0 kB/s 2024-04-26T20:10:46.399 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:46.431 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:46.556 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:46.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:46.697 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:46.731 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269338 s, 0.0 kB/s 2024-04-26T20:10:46.766 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:46.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:46.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:46.990 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:47.169 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:47.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:47.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222689 s, 0.0 kB/s 2024-04-26T20:10:47.283 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:47.306 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:47.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:47.448 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:47.529 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:47.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174825 s, 0.0 kB/s 2024-04-26T20:10:47.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:47.675 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:47.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:47.857 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:47.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:47.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:48.015 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000281346 s, 0.0 kB/s 2024-04-26T20:10:48.041 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:48.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:48.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:48.214 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:48.305 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:48.374 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:48.416 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157627 s, 0.0 kB/s 2024-04-26T20:10:48.478 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:48.524 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:48.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:48.649 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:48.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:48.835 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:48.885 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211557 s, 0.0 kB/s 2024-04-26T20:10:48.937 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:49.027 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:49.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:49.161 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:49.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:49.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:49.361 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221377 s, 0.0 kB/s 2024-04-26T20:10:49.437 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:49.486 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:49.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:49.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:49.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:49.861 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:49.911 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307591 s, 0.0 kB/s 2024-04-26T20:10:49.984 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:50.036 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:50.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:50.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:50.361 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:50.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:50.445 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268929 s, 0.0 kB/s 2024-04-26T20:10:50.496 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:50.545 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:50.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:50.686 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:50.836 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:50.887 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194872 s, 0.0 kB/s 2024-04-26T20:10:50.951 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:51.081 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:51.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:51.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:51.400 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:51.454 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000305644 s, 0.0 kB/s 2024-04-26T20:10:51.555 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:51.603 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:51.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:51.816 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:51.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:52.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:52.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173598 s, 0.0 kB/s 2024-04-26T20:10:52.147 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:52.195 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:52.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:52.375 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:52.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:52.521 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279178 s, 0.0 kB/s 2024-04-26T20:10:52.672 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:52.748 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:52.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:52.888 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:53.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:53.091 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:53.140 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264525 s, 0.0 kB/s 2024-04-26T20:10:53.191 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:53.238 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:53.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:53.388 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:53.506 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:53.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:53.598 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206276 s, 0.0 kB/s 2024-04-26T20:10:53.639 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:53.688 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:53.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:53.855 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:53.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:54.005 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017497 s, 0.0 kB/s 2024-04-26T20:10:54.139 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:54.188 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:54.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:54.338 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:54.461 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:54.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:54.572 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00028467 s, 0.0 kB/s 2024-04-26T20:10:54.626 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:54.672 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:54.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:54.847 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:54.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:55.022 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:55.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225488 s, 0.0 kB/s 2024-04-26T20:10:55.123 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:55.172 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:55.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:55.331 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:55.531 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207495 s, 0.0 kB/s 2024-04-26T20:10:55.582 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:55.631 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:55.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:55.831 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:55.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:55.997 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:56.064 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272835 s, 0.0 kB/s 2024-04-26T20:10:56.116 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:56.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:56.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:56.348 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:56.460 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:56.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:56.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188275 s, 0.0 kB/s 2024-04-26T20:10:56.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:56.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:56.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:56.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:56.981 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:57.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:57.065 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174215 s, 0.0 kB/s 2024-04-26T20:10:57.116 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:57.223 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:57.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:57.365 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:57.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:57.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:57.665 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178637 s, 0.0 kB/s 2024-04-26T20:10:57.717 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:57.765 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:57.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:57.926 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:58.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:58.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:58.124 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000291201 s, 0.0 kB/s 2024-04-26T20:10:58.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:58.224 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:58.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:58.374 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:58.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:58.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:58.632 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181916 s, 0.0 kB/s 2024-04-26T20:10:58.716 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:58.774 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:58.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:58.925 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:59.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251281 s, 0.0 kB/s 2024-04-26T20:10:59.176 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:59.225 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:59.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:59.367 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:10:59.539 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:10:59.592 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189265 s, 0.0 kB/s 2024-04-26T20:10:59.673 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:10:59.717 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:10:59.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:10:59.867 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:10:59.989 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211239 s, 0.0 kB/s 2024-04-26T20:11:00.160 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:00.209 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:00.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:00.359 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:00.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:00.531 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:00.601 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195226 s, 0.0 kB/s 2024-04-26T20:11:00.652 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:00.692 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:00.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:00.918 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:01.050 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:01.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197944 s, 0.0 kB/s 2024-04-26T20:11:01.234 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:01.268 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:01.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:01.424 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:01.527 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023252 s, 0.0 kB/s 2024-04-26T20:11:01.676 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:01.735 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:01.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:01.926 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:02.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:02.068 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:02.162 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192658 s, 0.0 kB/s 2024-04-26T20:11:02.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:02.243 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:02.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:02.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:02.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:02.577 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000353396 s, 0.0 kB/s 2024-04-26T20:11:02.612 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:02.644 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:02.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:02.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:02.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:02.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:03.019 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172082 s, 0.0 kB/s 2024-04-26T20:11:03.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:03.119 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:03.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:03.319 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:03.419 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:03.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:03.478 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030923 s, 0.0 kB/s 2024-04-26T20:11:03.512 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:03.544 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:03.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:03.678 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:03.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:03.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029691 s, 0.0 kB/s 2024-04-26T20:11:03.937 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:03.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:04.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:04.162 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:04.260 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:04.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:04.328 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214551 s, 0.0 kB/s 2024-04-26T20:11:04.363 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:04.421 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:04.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:04.545 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:04.646 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:04.704 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225196 s, 0.0 kB/s 2024-04-26T20:11:04.788 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:04.820 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:04.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:05.012 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182117 s, 0.0 kB/s 2024-04-26T20:11:05.205 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:05.262 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:05.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:05.479 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:05.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:05.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:05.696 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228417 s, 0.0 kB/s 2024-04-26T20:11:05.731 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:05.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:05.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:05.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:06.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:06.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:06.158 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197929 s, 0.0 kB/s 2024-04-26T20:11:06.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:06.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:06.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:06.410 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:06.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:06.608 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:06.664 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000311723 s, 0.0 kB/s 2024-04-26T20:11:06.747 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:06.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:06.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:06.938 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:07.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:07.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:07.255 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018843 s, 0.0 kB/s 2024-04-26T20:11:07.307 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:07.356 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:07.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:07.514 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:07.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:07.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:07.840 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248611 s, 0.0 kB/s 2024-04-26T20:11:07.874 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:07.923 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:08.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:08.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:08.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194827 s, 0.0 kB/s 2024-04-26T20:11:08.324 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:08.373 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:08.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:08.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:08.673 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:08.723 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186048 s, 0.0 kB/s 2024-04-26T20:11:08.789 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:08.823 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:08.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:09.040 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:09.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:09.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:09.240 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181229 s, 0.0 kB/s 2024-04-26T20:11:09.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:09.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:09.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:09.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:09.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:09.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:09.690 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178819 s, 0.0 kB/s 2024-04-26T20:11:09.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:09.867 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:09.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:10.007 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:10.123 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:10.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:10.208 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242026 s, 0.0 kB/s 2024-04-26T20:11:10.259 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:10.307 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:10.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:10.516 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:10.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:10.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:10.716 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020098 s, 0.0 kB/s 2024-04-26T20:11:10.801 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:10.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:10.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:11.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:11.159 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:11.241 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197066 s, 0.0 kB/s 2024-04-26T20:11:11.292 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:11.341 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:11.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:11.492 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:11.608 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:11.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226696 s, 0.0 kB/s 2024-04-26T20:11:11.784 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:11.833 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:11.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:12.009 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:12.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172527 s, 0.0 kB/s 2024-04-26T20:11:12.319 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:12.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:12.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:12.534 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:12.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:12.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199398 s, 0.0 kB/s 2024-04-26T20:11:12.802 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:12.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:13.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:13.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:13.276 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198602 s, 0.0 kB/s 2024-04-26T20:11:13.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:13.392 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:13.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:13.543 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:13.660 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:13.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165101 s, 0.0 kB/s 2024-04-26T20:11:13.835 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:13.868 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:13.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:14.026 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:14.147 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:14.176 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:14.226 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216779 s, 0.0 kB/s 2024-04-26T20:11:14.339 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:14.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:14.535 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:14.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:14.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:14.765 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148072 s, 0.0 kB/s 2024-04-26T20:11:14.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:14.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:14.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:15.010 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:15.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:15.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:15.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221124 s, 0.0 kB/s 2024-04-26T20:11:15.278 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:11:15.319 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:15.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:15.401 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:11:15.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:15.544 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:15.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:15.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:15.708 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185289 s, 0.0 kB/s 2024-04-26T20:11:15.785 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:15.809 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:15.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:15.959 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:16.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:16.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:16.142 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210276 s, 0.0 kB/s 2024-04-26T20:11:16.180 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:16.218 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:16.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:16.359 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:16.470 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:16.509 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:16.553 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210983 s, 0.0 kB/s 2024-04-26T20:11:16.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:16.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:16.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:16.785 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:16.957 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000199857 s, 0.0 kB/s 2024-04-26T20:11:17.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:17.043 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:17.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:17.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:17.291 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:17.336 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:17.377 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00028027 s, 0.0 kB/s 2024-04-26T20:11:17.411 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:17.477 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:17.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:17.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:17.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:17.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:17.895 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269393 s, 0.0 kB/s 2024-04-26T20:11:17.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:17.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:18.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:18.185 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:18.285 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:18.311 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:18.405 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016369 s, 0.0 kB/s 2024-04-26T20:11:18.445 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:18.478 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:18.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:18.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:18.703 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:18.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:18.770 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180374 s, 0.0 kB/s 2024-04-26T20:11:18.808 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:18.903 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:18.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:19.047 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:19.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:19.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219507 s, 0.0 kB/s 2024-04-26T20:11:19.296 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:19.345 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:19.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:19.511 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:19.628 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:19.662 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:19.712 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000210878 s, 0.0 kB/s 2024-04-26T20:11:19.788 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:19.837 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:19.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:19.987 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:20.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:20.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:20.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278702 s, 0.0 kB/s 2024-04-26T20:11:20.239 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:20.287 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:20.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:20.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:20.704 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:20.754 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179505 s, 0.0 kB/s 2024-04-26T20:11:20.805 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:20.854 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:20.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:21.004 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:21.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:21.162 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:21.213 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198256 s, 0.0 kB/s 2024-04-26T20:11:21.264 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:21.337 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:21.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:21.540 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:21.653 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:21.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:21.738 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193017 s, 0.0 kB/s 2024-04-26T20:11:21.789 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:21.851 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:21.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:21.997 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:22.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:22.171 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:22.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259695 s, 0.0 kB/s 2024-04-26T20:11:22.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:22.321 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:22.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:22.472 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:22.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:22.672 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026784 s, 0.0 kB/s 2024-04-26T20:11:22.773 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:22.822 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:22.972 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:23.087 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:23.130 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:23.180 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196661 s, 0.0 kB/s 2024-04-26T20:11:23.259 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:23.364 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:23.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:23.514 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:23.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:23.689 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252837 s, 0.0 kB/s 2024-04-26T20:11:23.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:23.839 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:23.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:23.990 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:24.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:24.198 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176401 s, 0.0 kB/s 2024-04-26T20:11:24.299 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:24.348 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:24.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:24.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:24.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:24.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:24.698 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227918 s, 0.0 kB/s 2024-04-26T20:11:24.750 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:24.798 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:24.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:24.948 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:25.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:25.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001982 s, 0.0 kB/s 2024-04-26T20:11:25.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:25.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:25.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:25.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:25.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:25.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:25.757 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023177 s, 0.0 kB/s 2024-04-26T20:11:25.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:25.841 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:25.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:26.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:26.132 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:26.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022388 s, 0.0 kB/s 2024-04-26T20:11:26.275 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:26.324 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:26.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:26.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:26.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:26.650 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:26.699 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179476 s, 0.0 kB/s 2024-04-26T20:11:26.751 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:26.858 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:27.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:27.058 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:27.182 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:27.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:27.266 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190839 s, 0.0 kB/s 2024-04-26T20:11:27.318 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:27.366 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:27.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:27.517 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:27.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:27.759 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200106 s, 0.0 kB/s 2024-04-26T20:11:27.810 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:27.850 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:27.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:28.017 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:28.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:28.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:28.200 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169871 s, 0.0 kB/s 2024-04-26T20:11:28.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:28.292 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:28.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:28.417 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:28.524 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:28.559 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:28.601 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188247 s, 0.0 kB/s 2024-04-26T20:11:28.644 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:28.719 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:28.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:28.842 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:28.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:29.034 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:29.068 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216391 s, 0.0 kB/s 2024-04-26T20:11:29.102 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:29.185 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:29.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:29.351 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:29.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:29.493 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215375 s, 0.0 kB/s 2024-04-26T20:11:29.577 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:29.610 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:29.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:29.735 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:29.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:29.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:29.910 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172569 s, 0.0 kB/s 2024-04-26T20:11:29.953 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:30.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:30.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:30.143 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:30.243 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:30.299 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:30.335 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000306251 s, 0.0 kB/s 2024-04-26T20:11:30.370 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:30.402 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:30.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:30.527 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:30.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:30.661 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:30.744 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000302638 s, 0.0 kB/s 2024-04-26T20:11:30.789 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:30.891 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:31.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:31.069 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:31.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:31.319 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197385 s, 0.0 kB/s 2024-04-26T20:11:31.404 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:31.453 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:31.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:31.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:31.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234071 s, 0.0 kB/s 2024-04-26T20:11:31.871 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:31.920 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:32.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:32.070 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:32.186 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:32.236 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:32.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018557 s, 0.0 kB/s 2024-04-26T20:11:32.338 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:32.390 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:32.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:32.528 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:32.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:32.695 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:32.745 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000250865 s, 0.0 kB/s 2024-04-26T20:11:32.796 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:32.845 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:32.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:32.995 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:33.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:33.302 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000289811 s, 0.0 kB/s 2024-04-26T20:11:33.355 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:33.404 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:33.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:33.653 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:33.770 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:33.804 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:33.892 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252312 s, 0.0 kB/s 2024-04-26T20:11:33.931 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:33.979 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:34.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:34.163 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:34.278 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:34.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:34.363 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168792 s, 0.0 kB/s 2024-04-26T20:11:34.414 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:34.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:34.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:34.663 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:34.781 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:34.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:34.888 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268459 s, 0.0 kB/s 2024-04-26T20:11:34.940 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:34.988 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:35.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:35.172 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:35.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:35.421 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258266 s, 0.0 kB/s 2024-04-26T20:11:35.473 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:35.522 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:35.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:35.672 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:35.829 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:35.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:35.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248658 s, 0.0 kB/s 2024-04-26T20:11:35.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:36.030 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:36.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:36.189 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:36.306 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:36.392 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:36.439 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178556 s, 0.0 kB/s 2024-04-26T20:11:36.491 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:36.539 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:36.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:36.689 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:36.814 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:36.921 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:36.973 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237773 s, 0.0 kB/s 2024-04-26T20:11:37.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:37.142 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:37.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:37.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:37.524 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:37.557 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:37.620 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197745 s, 0.0 kB/s 2024-04-26T20:11:37.658 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:37.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:37.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:37.857 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:38.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:38.057 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188392 s, 0.0 kB/s 2024-04-26T20:11:38.111 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:38.157 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:38.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:38.332 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:38.582 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226519 s, 0.0 kB/s 2024-04-26T20:11:38.634 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:38.682 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:38.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:38.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:38.999 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:39.099 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:39.149 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197632 s, 0.0 kB/s 2024-04-26T20:11:39.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:39.249 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:39.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:39.399 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:39.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:39.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:39.633 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287286 s, 0.0 kB/s 2024-04-26T20:11:39.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:39.749 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:39.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:39.891 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:40.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:40.124 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162688 s, 0.0 kB/s 2024-04-26T20:11:40.275 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:40.325 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:40.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:40.493 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:40.618 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:40.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177664 s, 0.0 kB/s 2024-04-26T20:11:40.778 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:40.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:40.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:41.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:41.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:41.177 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:41.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213096 s, 0.0 kB/s 2024-04-26T20:11:41.279 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:41.327 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:41.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:41.503 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:41.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:41.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:41.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150056 s, 0.0 kB/s 2024-04-26T20:11:41.854 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:41.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:42.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:42.078 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:42.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:42.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221181 s, 0.0 kB/s 2024-04-26T20:11:42.337 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:42.386 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:42.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:42.561 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:42.726 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:42.969 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000130317 s, 0.0 kB/s 2024-04-26T20:11:43.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:43.078 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:43.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:43.261 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:43.445 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:43.495 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157993 s, 0.0 kB/s 2024-04-26T20:11:43.605 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:43.653 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:43.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:43.804 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:43.970 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001791 s, 0.0 kB/s 2024-04-26T20:11:44.096 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:44.162 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:44.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:44.362 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:44.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:44.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:44.662 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187448 s, 0.0 kB/s 2024-04-26T20:11:44.736 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:44.796 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:44.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:45.004 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:45.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:45.204 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:45.271 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196876 s, 0.0 kB/s 2024-04-26T20:11:45.339 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:45.404 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:45.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:45.580 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:45.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:11:45.776 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:11:45.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:45.896 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:45.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186141 s, 0.0 kB/s 2024-04-26T20:11:46.030 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:46.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:46.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:46.330 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:46.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:46.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:46.597 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000289986 s, 0.0 kB/s 2024-04-26T20:11:46.665 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:46.730 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:46.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:46.964 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:47.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:47.155 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:47.214 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247961 s, 0.0 kB/s 2024-04-26T20:11:47.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:47.330 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:47.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:47.506 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:47.631 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:47.756 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196592 s, 0.0 kB/s 2024-04-26T20:11:47.815 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:47.873 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:48.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:48.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:48.197 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:48.306 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174836 s, 0.0 kB/s 2024-04-26T20:11:48.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:48.488 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:48.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:48.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:48.772 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:48.856 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000501116 s, 0.0 kB/s 2024-04-26T20:11:48.958 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:49.020 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:49.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:49.173 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:49.308 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:49.365 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:49.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000254077 s, 0.0 kB/s 2024-04-26T20:11:49.466 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:49.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:49.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:49.665 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:49.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:49.898 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177798 s, 0.0 kB/s 2024-04-26T20:11:50.000 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:50.049 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:50.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:50.199 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:50.315 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:50.349 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150214 s, 0.0 kB/s 2024-04-26T20:11:50.450 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:50.510 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:50.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:50.716 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:50.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:50.891 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:50.941 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197116 s, 0.0 kB/s 2024-04-26T20:11:50.992 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:51.106 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:51.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:51.241 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:51.359 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:51.481 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:51.533 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205459 s, 0.0 kB/s 2024-04-26T20:11:51.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:51.633 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:51.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:51.783 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:51.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:51.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:52.008 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255145 s, 0.0 kB/s 2024-04-26T20:11:52.059 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:52.108 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:52.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:52.312 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:52.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:52.500 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:52.550 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000270664 s, 0.0 kB/s 2024-04-26T20:11:52.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:52.650 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:52.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:52.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:52.958 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:53.008 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026561 s, 0.0 kB/s 2024-04-26T20:11:53.060 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:53.109 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:53.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:53.300 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:53.419 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:53.450 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:53.577 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218602 s, 0.0 kB/s 2024-04-26T20:11:53.610 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:53.659 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:53.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:53.809 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:53.927 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:53.992 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213702 s, 0.0 kB/s 2024-04-26T20:11:54.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:54.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:54.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:54.326 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:54.501 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:54.551 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238323 s, 0.0 kB/s 2024-04-26T20:11:54.603 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:54.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:54.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:54.801 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:54.918 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:55.019 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:55.068 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263244 s, 0.0 kB/s 2024-04-26T20:11:55.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:55.169 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:55.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:55.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:55.506 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:55.560 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:55.616 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231709 s, 0.0 kB/s 2024-04-26T20:11:55.661 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:55.710 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:55.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:55.885 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:56.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:56.144 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200523 s, 0.0 kB/s 2024-04-26T20:11:56.195 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:56.244 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:56.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:56.394 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:56.510 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:56.560 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:56.611 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205974 s, 0.0 kB/s 2024-04-26T20:11:56.662 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:56.727 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:56.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:56.911 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:57.026 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:57.061 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:57.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195433 s, 0.0 kB/s 2024-04-26T20:11:57.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:57.229 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:57.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:57.369 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:57.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:57.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017499 s, 0.0 kB/s 2024-04-26T20:11:57.654 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:57.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:57.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:57.911 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:58.025 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:58.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152203 s, 0.0 kB/s 2024-04-26T20:11:58.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:58.212 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:58.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:58.370 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:58.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:58.665 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246578 s, 0.0 kB/s 2024-04-26T20:11:58.713 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:58.762 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:58.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:58.980 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:59.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:59.179 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179536 s, 0.0 kB/s 2024-04-26T20:11:59.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:59.304 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:59.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:59.446 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:11:59.569 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:11:59.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:11:59.671 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161649 s, 0.0 kB/s 2024-04-26T20:11:59.722 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:11:59.830 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:11:59.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:11:59.971 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:00.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:00.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:00.171 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026298 s, 0.0 kB/s 2024-04-26T20:12:00.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:00.271 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:00.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:00.455 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:00.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:00.655 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:00.705 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245014 s, 0.0 kB/s 2024-04-26T20:12:00.756 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:00.813 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:00.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:00.963 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:01.103 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:01.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238917 s, 0.0 kB/s 2024-04-26T20:12:01.256 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:01.355 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:01.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:01.539 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:01.664 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:01.714 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:01.781 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187021 s, 0.0 kB/s 2024-04-26T20:12:01.848 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:01.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:02.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:02.114 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:02.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:02.322 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:02.447 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173429 s, 0.0 kB/s 2024-04-26T20:12:02.515 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:02.581 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:02.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:02.781 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:02.981 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:03.048 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225475 s, 0.0 kB/s 2024-04-26T20:12:03.116 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:03.181 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:03.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:03.381 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:03.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:03.664 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211081 s, 0.0 kB/s 2024-04-26T20:12:03.732 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:03.798 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:03.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:03.981 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:04.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212249 s, 0.0 kB/s 2024-04-26T20:12:04.283 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:04.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:04.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:04.507 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:04.844 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219827 s, 0.0 kB/s 2024-04-26T20:12:04.891 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:04.940 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:05.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:05.090 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:05.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:05.276 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:05.332 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219685 s, 0.0 kB/s 2024-04-26T20:12:05.439 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:05.491 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:05.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:05.695 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:05.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:05.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:05.907 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202883 s, 0.0 kB/s 2024-04-26T20:12:05.959 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:06.016 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:06.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:06.166 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:06.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:06.341 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:06.391 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206448 s, 0.0 kB/s 2024-04-26T20:12:06.484 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:06.574 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:06.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:06.716 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:06.833 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:06.866 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:06.916 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264801 s, 0.0 kB/s 2024-04-26T20:12:06.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:07.021 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:07.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:07.166 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:07.282 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:07.325 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:07.375 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169778 s, 0.0 kB/s 2024-04-26T20:12:07.426 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:07.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:07.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:07.650 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:07.825 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:07.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238343 s, 0.0 kB/s 2024-04-26T20:12:07.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:08.025 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:08.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:08.192 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:08.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:08.342 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:08.459 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251647 s, 0.0 kB/s 2024-04-26T20:12:08.510 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:08.559 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:08.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:08.709 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:08.867 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:08.917 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231644 s, 0.0 kB/s 2024-04-26T20:12:09.006 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:09.101 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:09.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:09.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:09.369 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:09.434 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:09.484 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180322 s, 0.0 kB/s 2024-04-26T20:12:09.536 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:09.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:09.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:09.734 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:09.852 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:09.976 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195094 s, 0.0 kB/s 2024-04-26T20:12:10.077 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:10.126 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:10.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:10.281 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:10.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:10.427 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234341 s, 0.0 kB/s 2024-04-26T20:12:10.453 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:10.550 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:10.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:10.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:10.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:10.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:10.777 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000334925 s, 0.0 kB/s 2024-04-26T20:12:10.834 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:10.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:10.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:10.994 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:11.127 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:11.213 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152337 s, 0.0 kB/s 2024-04-26T20:12:11.266 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:11.310 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:11.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:11.444 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:11.543 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:11.619 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179628 s, 0.0 kB/s 2024-04-26T20:12:11.662 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:11.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:11.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:11.919 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:12.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:12.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:12.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019517 s, 0.0 kB/s 2024-04-26T20:12:12.145 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:12.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:12.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:12.396 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:12.541 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:12.662 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294519 s, 0.0 kB/s 2024-04-26T20:12:12.700 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:12.737 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:12.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:12.853 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:12.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:12.979 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:13.012 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237662 s, 0.0 kB/s 2024-04-26T20:12:13.046 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:13.079 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:13.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:13.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:13.418 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:13.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183935 s, 0.0 kB/s 2024-04-26T20:12:13.488 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:13.540 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:13.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:13.654 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:13.753 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:13.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:13.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221086 s, 0.0 kB/s 2024-04-26T20:12:13.930 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:13.963 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:14.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:14.121 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:14.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:14.263 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:14.296 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197067 s, 0.0 kB/s 2024-04-26T20:12:14.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:14.413 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:14.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:14.563 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:14.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:14.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:14.780 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000151585 s, 0.0 kB/s 2024-04-26T20:12:14.814 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:14.846 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:14.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:14.974 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:15.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:15.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:15.147 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167947 s, 0.0 kB/s 2024-04-26T20:12:15.181 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:15.213 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:15.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:15.397 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:15.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:15.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:15.600 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000239239 s, 0.0 kB/s 2024-04-26T20:12:15.632 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:15.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:15.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:15.847 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:15.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:15.972 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:16.006 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280059 s, 0.0 kB/s 2024-04-26T20:12:16.041 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:16.133 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:16.192 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:12:16.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:16.264 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:16.299 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:12:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:16.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:16.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197699 s, 0.0 kB/s 2024-04-26T20:12:16.610 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:16.648 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:16.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:16.818 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:16.956 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:16.990 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150986 s, 0.0 kB/s 2024-04-26T20:12:17.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:17.057 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:17.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:17.182 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:17.285 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:17.307 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:17.340 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191093 s, 0.0 kB/s 2024-04-26T20:12:17.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:17.516 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:17.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:17.649 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:17.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:17.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:17.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183135 s, 0.0 kB/s 2024-04-26T20:12:17.850 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:17.900 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:18.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:18.049 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:18.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:18.241 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219485 s, 0.0 kB/s 2024-04-26T20:12:18.286 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:18.333 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:18.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:18.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:18.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:18.769 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191356 s, 0.0 kB/s 2024-04-26T20:12:18.809 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:18.851 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:18.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:18.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:19.088 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:19.125 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:19.159 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155775 s, 0.0 kB/s 2024-04-26T20:12:19.193 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:19.225 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:19.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:19.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:19.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:19.623 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237375 s, 0.0 kB/s 2024-04-26T20:12:19.660 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:19.692 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:19.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:19.809 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:19.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:19.942 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:19.995 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029607 s, 0.0 kB/s 2024-04-26T20:12:20.077 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:20.109 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:20.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:20.243 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:20.339 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:20.405 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192687 s, 0.0 kB/s 2024-04-26T20:12:20.436 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:20.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:20.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:20.651 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:20.750 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:20.785 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:20.818 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176548 s, 0.0 kB/s 2024-04-26T20:12:20.869 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:20.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:20.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:21.026 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:21.161 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:21.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:21.227 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000193915 s, 0.0 kB/s 2024-04-26T20:12:21.261 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:21.293 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:21.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:21.460 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:21.594 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:21.695 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:21.727 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232848 s, 0.0 kB/s 2024-04-26T20:12:21.762 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:21.828 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:21.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:21.961 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:22.066 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:22.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:22.169 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262703 s, 0.0 kB/s 2024-04-26T20:12:22.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:22.243 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:22.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:22.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:22.494 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000234184 s, 0.0 kB/s 2024-04-26T20:12:22.571 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:22.687 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:22.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:22.861 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:22.959 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:23.105 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176295 s, 0.0 kB/s 2024-04-26T20:12:23.146 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:23.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:23.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:23.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:23.407 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:23.428 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:23.462 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261577 s, 0.0 kB/s 2024-04-26T20:12:23.496 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:23.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:23.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:23.712 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:23.887 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:23.920 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000307606 s, 0.0 kB/s 2024-04-26T20:12:23.982 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:24.004 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:24.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:24.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:24.313 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:24.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:24.354 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206683 s, 0.0 kB/s 2024-04-26T20:12:24.381 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:24.428 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:24.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:24.538 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:24.630 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:24.698 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025198 s, 0.0 kB/s 2024-04-26T20:12:24.756 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:24.886 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:24.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:24.996 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:25.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:25.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:25.171 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226995 s, 0.0 kB/s 2024-04-26T20:12:25.207 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:25.238 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:25.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:25.372 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:25.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:25.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000287769 s, 0.0 kB/s 2024-04-26T20:12:25.706 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:25.739 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:25.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:25.914 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:26.017 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:26.099 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176356 s, 0.0 kB/s 2024-04-26T20:12:26.132 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:26.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:26.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:26.347 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:26.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:26.506 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245777 s, 0.0 kB/s 2024-04-26T20:12:26.541 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:26.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:26.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:26.698 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:26.799 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:26.823 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:26.885 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170354 s, 0.0 kB/s 2024-04-26T20:12:26.924 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:26.973 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:27.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:27.223 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:27.336 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:27.394 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:27.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190717 s, 0.0 kB/s 2024-04-26T20:12:27.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:27.491 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:27.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:27.590 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:27.683 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:27.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:27.732 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219658 s, 0.0 kB/s 2024-04-26T20:12:27.750 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:27.765 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:27.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:27.923 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:28.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:28.065 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:28.082 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021151 s, 0.0 kB/s 2024-04-26T20:12:28.159 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:28.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:28.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:28.307 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:28.387 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:28.408 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:28.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223943 s, 0.0 kB/s 2024-04-26T20:12:28.442 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:28.458 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:28.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:28.583 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:28.667 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:28.765 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:28.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171332 s, 0.0 kB/s 2024-04-26T20:12:28.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:28.888 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:28.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:29.003 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:29.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:29.109 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213292 s, 0.0 kB/s 2024-04-26T20:12:29.127 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:29.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:29.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:29.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:29.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:29.420 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:29.470 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235515 s, 0.0 kB/s 2024-04-26T20:12:29.522 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:29.602 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:29.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:29.746 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:29.860 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:29.974 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000417595 s, 0.0 kB/s 2024-04-26T20:12:30.130 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:30.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:30.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:30.321 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:30.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:30.471 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:30.596 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278546 s, 0.0 kB/s 2024-04-26T20:12:30.647 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:30.696 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:30.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:30.842 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:31.163 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:31.213 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180461 s, 0.0 kB/s 2024-04-26T20:12:31.264 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:31.349 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:31.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:31.546 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:31.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:31.696 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:31.746 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224794 s, 0.0 kB/s 2024-04-26T20:12:31.798 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:31.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:32.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:32.047 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:32.162 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:32.222 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:32.287 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252597 s, 0.0 kB/s 2024-04-26T20:12:32.347 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:32.388 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:32.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:32.539 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:32.655 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:32.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:32.738 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181937 s, 0.0 kB/s 2024-04-26T20:12:32.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:32.864 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:32.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:33.014 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:33.132 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:33.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178661 s, 0.0 kB/s 2024-04-26T20:12:33.273 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:33.322 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:33.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:33.472 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:33.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:33.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:33.690 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022914 s, 0.0 kB/s 2024-04-26T20:12:33.766 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:33.814 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:33.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:33.964 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:34.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:34.186 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00028327 s, 0.0 kB/s 2024-04-26T20:12:34.241 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:34.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:34.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:34.440 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:34.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:34.641 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000316731 s, 0.0 kB/s 2024-04-26T20:12:34.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:34.840 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:34.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:35.015 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:35.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:35.240 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000230818 s, 0.0 kB/s 2024-04-26T20:12:35.291 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:35.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:35.490 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:35.654 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:35.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:35.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019358 s, 0.0 kB/s 2024-04-26T20:12:35.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:35.840 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:36.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:36.049 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:36.165 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:36.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247926 s, 0.0 kB/s 2024-04-26T20:12:36.316 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:36.349 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:36.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:36.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:36.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:36.754 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:36.799 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194738 s, 0.0 kB/s 2024-04-26T20:12:36.850 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:36.899 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:37.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:37.074 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:37.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:37.274 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259433 s, 0.0 kB/s 2024-04-26T20:12:37.326 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:37.400 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:37.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:37.583 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:37.710 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:37.791 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205359 s, 0.0 kB/s 2024-04-26T20:12:37.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:37.891 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:37.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:38.041 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:38.158 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:38.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227196 s, 0.0 kB/s 2024-04-26T20:12:38.284 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:38.382 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:38.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:38.525 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:38.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:38.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000359188 s, 0.0 kB/s 2024-04-26T20:12:38.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:38.875 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:38.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:39.025 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:39.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:39.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155034 s, 0.0 kB/s 2024-04-26T20:12:39.327 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:39.359 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:39.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:39.476 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:39.578 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:39.614 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:39.684 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170354 s, 0.0 kB/s 2024-04-26T20:12:39.750 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:39.776 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:39.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:39.993 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:40.091 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153354 s, 0.0 kB/s 2024-04-26T20:12:40.186 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:40.218 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:40.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:40.335 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:40.468 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185769 s, 0.0 kB/s 2024-04-26T20:12:40.592 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:40.619 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:40.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:40.752 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:40.869 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:40.902 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175937 s, 0.0 kB/s 2024-04-26T20:12:40.937 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:40.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:41.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:41.086 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:41.186 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:41.211 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:41.244 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219976 s, 0.0 kB/s 2024-04-26T20:12:41.279 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:41.311 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:41.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:41.564 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:41.686 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:41.740 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263879 s, 0.0 kB/s 2024-04-26T20:12:41.787 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:41.836 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:41.953 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:42.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:42.112 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:42.162 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192533 s, 0.0 kB/s 2024-04-26T20:12:42.213 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:42.262 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:42.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:42.412 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:42.525 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:42.562 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146404 s, 0.0 kB/s 2024-04-26T20:12:42.671 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:42.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:42.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:42.904 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:43.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:43.204 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208879 s, 0.0 kB/s 2024-04-26T20:12:43.255 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:43.304 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:43.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:43.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:43.571 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:43.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:43.671 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244436 s, 0.0 kB/s 2024-04-26T20:12:43.722 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:43.771 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:43.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:43.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:44.062 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:44.121 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:44.171 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162114 s, 0.0 kB/s 2024-04-26T20:12:44.222 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:44.271 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:44.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:44.421 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:44.542 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:44.629 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263927 s, 0.0 kB/s 2024-04-26T20:12:44.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:44.771 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:44.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:44.921 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:45.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:45.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:45.147 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220479 s, 0.0 kB/s 2024-04-26T20:12:45.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:45.247 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:45.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:45.397 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:45.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:45.630 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198489 s, 0.0 kB/s 2024-04-26T20:12:45.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:45.730 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:45.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:45.880 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:45.995 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:46.030 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:46.080 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156442 s, 0.0 kB/s 2024-04-26T20:12:46.132 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:46.243 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:46.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:46.381 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:46.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:46.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:46.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235592 s, 0.0 kB/s 2024-04-26T20:12:46.681 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:12:46.732 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:46.781 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:46.832 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:12:46.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:46.981 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:47.105 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:47.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:47.206 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019026 s, 0.0 kB/s 2024-04-26T20:12:47.257 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:47.357 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:47.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:47.531 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:47.649 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:47.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:47.741 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191307 s, 0.0 kB/s 2024-04-26T20:12:47.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:47.840 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:47.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:47.990 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:48.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:48.190 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185939 s, 0.0 kB/s 2024-04-26T20:12:48.292 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:48.340 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:48.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:48.499 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:48.615 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:48.671 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:48.767 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000253181 s, 0.0 kB/s 2024-04-26T20:12:48.825 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:48.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:49.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:49.049 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:49.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:49.272 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015008 s, 0.0 kB/s 2024-04-26T20:12:49.375 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:49.424 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:49.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:49.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:49.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:49.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:49.791 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214421 s, 0.0 kB/s 2024-04-26T20:12:49.842 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:49.892 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:50.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:50.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:50.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:50.266 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:50.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182185 s, 0.0 kB/s 2024-04-26T20:12:50.393 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:50.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:50.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:50.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:50.710 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:50.792 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271844 s, 0.0 kB/s 2024-04-26T20:12:50.876 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:50.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:51.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:51.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:51.213 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:51.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00038752 s, 0.0 kB/s 2024-04-26T20:12:51.382 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:51.426 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:51.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:51.576 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:51.692 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:51.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:51.826 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241727 s, 0.0 kB/s 2024-04-26T20:12:51.877 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:51.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:52.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:52.076 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:52.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:52.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:52.301 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169216 s, 0.0 kB/s 2024-04-26T20:12:52.391 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:52.426 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:52.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:52.576 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:52.693 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:52.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:52.868 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164547 s, 0.0 kB/s 2024-04-26T20:12:52.919 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:52.960 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:53.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:53.123 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:53.224 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:53.260 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:53.293 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216215 s, 0.0 kB/s 2024-04-26T20:12:53.336 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:53.368 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:53.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:53.543 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:53.651 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:53.762 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000167649 s, 0.0 kB/s 2024-04-26T20:12:53.787 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:53.828 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:53.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:53.977 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:54.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:54.102 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:54.136 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154329 s, 0.0 kB/s 2024-04-26T20:12:54.171 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:54.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:54.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:54.386 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:54.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:54.519 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:54.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244325 s, 0.0 kB/s 2024-04-26T20:12:54.654 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:54.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:54.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:54.845 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:54.956 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:55.011 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:55.045 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000302047 s, 0.0 kB/s 2024-04-26T20:12:55.080 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:55.111 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:55.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:55.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:55.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:55.362 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:55.395 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160697 s, 0.0 kB/s 2024-04-26T20:12:55.430 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:55.462 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:55.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:55.728 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:55.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:55.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:55.935 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184058 s, 0.0 kB/s 2024-04-26T20:12:55.987 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:56.035 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:56.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:56.202 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:56.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:56.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233097 s, 0.0 kB/s 2024-04-26T20:12:56.487 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:56.536 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:56.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:56.686 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:56.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:56.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:56.894 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000138198 s, 0.0 kB/s 2024-04-26T20:12:56.946 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:56.994 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:57.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:57.144 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:57.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:57.295 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:57.344 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016879 s, 0.0 kB/s 2024-04-26T20:12:57.396 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:57.461 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:57.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:57.611 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:57.728 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:57.761 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016326 s, 0.0 kB/s 2024-04-26T20:12:57.863 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:57.943 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:58.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:58.148 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:58.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:58.320 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:58.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202709 s, 0.0 kB/s 2024-04-26T20:12:58.471 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:58.545 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:58.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:58.720 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:58.836 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:58.870 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:58.920 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279968 s, 0.0 kB/s 2024-04-26T20:12:58.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:59.020 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:59.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:59.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:59.314 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:59.362 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:59.412 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175359 s, 0.0 kB/s 2024-04-26T20:12:59.464 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:59.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:12:59.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:12:59.662 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:12:59.779 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:12:59.812 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:12:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194435 s, 0.0 kB/s 2024-04-26T20:12:59.914 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:12:59.963 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:00.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:00.120 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:00.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:00.296 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:00.346 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209441 s, 0.0 kB/s 2024-04-26T20:13:00.457 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:00.496 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:00.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:00.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:00.798 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:00.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:00.905 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020751 s, 0.0 kB/s 2024-04-26T20:13:00.956 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:01.013 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:01.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:01.213 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:01.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:01.364 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:01.413 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304765 s, 0.0 kB/s 2024-04-26T20:13:01.465 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:01.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:01.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:01.708 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:01.830 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:01.855 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:01.960 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190752 s, 0.0 kB/s 2024-04-26T20:13:02.007 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:02.105 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:02.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:02.289 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:02.405 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:02.439 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:02.489 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000259901 s, 0.0 kB/s 2024-04-26T20:13:02.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:02.589 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:02.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:02.739 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:02.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:02.897 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00027851 s, 0.0 kB/s 2024-04-26T20:13:03.041 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:03.089 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:03.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:03.248 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:03.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:03.423 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:03.473 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222237 s, 0.0 kB/s 2024-04-26T20:13:03.524 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:03.573 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:03.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:03.723 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:03.890 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:03.940 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189451 s, 0.0 kB/s 2024-04-26T20:13:03.991 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:04.040 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:04.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:04.215 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:04.334 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:04.365 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217308 s, 0.0 kB/s 2024-04-26T20:13:04.467 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:04.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:04.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:04.657 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:04.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:04.874 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:04.924 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252984 s, 0.0 kB/s 2024-04-26T20:13:04.975 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:05.024 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:05.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:05.174 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:05.289 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:05.424 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00021107 s, 0.0 kB/s 2024-04-26T20:13:05.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:05.533 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:05.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:05.691 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:05.806 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:05.858 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:05.908 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237837 s, 0.0 kB/s 2024-04-26T20:13:05.960 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:06.008 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:06.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:06.167 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:06.265 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:06.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186848 s, 0.0 kB/s 2024-04-26T20:13:06.401 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:06.498 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:06.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:06.609 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:06.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:06.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:06.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304824 s, 0.0 kB/s 2024-04-26T20:13:06.868 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:06.900 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:06.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:07.026 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:07.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:07.226 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:07.259 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226172 s, 0.0 kB/s 2024-04-26T20:13:07.294 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:07.326 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:07.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:07.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:07.599 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187688 s, 0.0 kB/s 2024-04-26T20:13:07.742 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:07.776 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:07.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:07.976 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:08.077 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:08.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:08.135 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185302 s, 0.0 kB/s 2024-04-26T20:13:08.169 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:08.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:08.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:08.318 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:08.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:08.452 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212083 s, 0.0 kB/s 2024-04-26T20:13:08.544 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:08.577 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:08.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:08.752 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:08.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:08.969 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284968 s, 0.0 kB/s 2024-04-26T20:13:09.070 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:09.139 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:09.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:09.285 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:09.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:09.452 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:09.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204869 s, 0.0 kB/s 2024-04-26T20:13:09.554 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:09.659 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:09.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:09.811 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:09.926 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:10.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235623 s, 0.0 kB/s 2024-04-26T20:13:10.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:10.145 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:10.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:10.294 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:10.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:10.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:10.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000146632 s, 0.0 kB/s 2024-04-26T20:13:10.589 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:10.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:10.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:10.786 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:10.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:10.936 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:11.027 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002637 s, 0.0 kB/s 2024-04-26T20:13:11.083 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:11.137 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:11.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:11.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:11.452 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:11.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:11.573 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153316 s, 0.0 kB/s 2024-04-26T20:13:11.630 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:11.679 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:11.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:11.829 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:11.947 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:12.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:12.054 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261678 s, 0.0 kB/s 2024-04-26T20:13:12.105 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:12.154 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:12.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:12.396 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:12.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:12.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:12.596 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00026176 s, 0.0 kB/s 2024-04-26T20:13:12.647 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:12.696 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:12.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:12.846 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:12.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:13.004 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:13.054 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211051 s, 0.0 kB/s 2024-04-26T20:13:13.114 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:13.163 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:13.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:13.329 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:13.449 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:13.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:13.579 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267702 s, 0.0 kB/s 2024-04-26T20:13:13.631 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:13.717 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:13.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:13.888 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:14.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:14.038 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:14.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000142508 s, 0.0 kB/s 2024-04-26T20:13:14.156 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:14.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:14.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:14.381 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:14.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:14.580 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185455 s, 0.0 kB/s 2024-04-26T20:13:14.631 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:14.745 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:14.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:14.872 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:15.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:15.114 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001592 s, 0.0 kB/s 2024-04-26T20:13:15.165 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:15.214 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:15.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:15.381 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:15.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:15.606 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154354 s, 0.0 kB/s 2024-04-26T20:13:15.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:15.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:15.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:15.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:16.023 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:16.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:16.148 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000155083 s, 0.0 kB/s 2024-04-26T20:13:16.234 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:16.273 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:16.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:16.423 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:16.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:16.573 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:16.625 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170933 s, 0.0 kB/s 2024-04-26T20:13:16.675 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:16.723 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:16.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:16.924 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:17.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:13:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:17.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:17.275 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000318746 s, 0.0 kB/s 2024-04-26T20:13:17.324 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:17.374 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:17.431 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:13:17.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:17.574 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:17.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:17.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:17.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264098 s, 0.0 kB/s 2024-04-26T20:13:17.867 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:17.916 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:18.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:18.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:18.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:18.250 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000360499 s, 0.0 kB/s 2024-04-26T20:13:18.376 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:18.491 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:18.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:18.624 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:18.827 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:18.883 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:18.956 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173316 s, 0.0 kB/s 2024-04-26T20:13:19.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:19.058 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:19.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:19.208 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:19.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:19.408 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185216 s, 0.0 kB/s 2024-04-26T20:13:19.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:19.517 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:19.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:19.692 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:19.866 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:19.917 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:19.967 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257567 s, 0.0 kB/s 2024-04-26T20:13:20.018 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:20.067 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:20.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:20.217 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:20.409 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:20.459 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192669 s, 0.0 kB/s 2024-04-26T20:13:20.511 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:20.584 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:20.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:20.751 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:20.867 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:20.951 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231095 s, 0.0 kB/s 2024-04-26T20:13:21.063 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:21.101 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:21.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:21.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:21.401 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:21.499 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174954 s, 0.0 kB/s 2024-04-26T20:13:21.544 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:21.593 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:21.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:21.766 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:21.999 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174238 s, 0.0 kB/s 2024-04-26T20:13:22.034 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:22.066 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:22.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:22.231 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:22.351 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:22.399 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:22.479 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157232 s, 0.0 kB/s 2024-04-26T20:13:22.517 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:22.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:22.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:22.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:22.776 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:22.800 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:22.858 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227996 s, 0.0 kB/s 2024-04-26T20:13:22.901 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:22.950 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:23.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:23.133 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:23.251 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:23.303 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:23.421 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168322 s, 0.0 kB/s 2024-04-26T20:13:23.460 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:23.509 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:23.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:23.659 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:23.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:23.925 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000286664 s, 0.0 kB/s 2024-04-26T20:13:23.977 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:24.026 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:24.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:24.176 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:24.290 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:24.376 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173022 s, 0.0 kB/s 2024-04-26T20:13:24.427 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:24.476 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:24.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:24.676 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:24.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:24.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:24.918 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271368 s, 0.0 kB/s 2024-04-26T20:13:24.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:25.018 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:25.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:25.168 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:25.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:25.335 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:25.385 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156717 s, 0.0 kB/s 2024-04-26T20:13:25.436 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:25.485 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:25.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:25.660 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:25.775 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:25.848 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:25.910 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024965 s, 0.0 kB/s 2024-04-26T20:13:25.961 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:26.029 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:26.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:26.193 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:26.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:26.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:26.393 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018602 s, 0.0 kB/s 2024-04-26T20:13:26.445 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:26.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:26.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:26.669 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:26.844 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:26.894 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217959 s, 0.0 kB/s 2024-04-26T20:13:26.945 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:27.038 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:27.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:27.194 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:27.325 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:27.369 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:27.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162416 s, 0.0 kB/s 2024-04-26T20:13:27.471 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:27.519 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:27.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:27.670 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:27.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:27.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:27.895 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251159 s, 0.0 kB/s 2024-04-26T20:13:27.946 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:27.995 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:28.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:28.195 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:28.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:28.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:28.453 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231683 s, 0.0 kB/s 2024-04-26T20:13:28.518 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:28.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:28.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:28.703 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:28.819 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:28.853 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000174482 s, 0.0 kB/s 2024-04-26T20:13:28.955 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:29.053 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:29.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:29.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:29.318 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:29.404 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160035 s, 0.0 kB/s 2024-04-26T20:13:29.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:29.562 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:29.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:29.724 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:29.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:29.912 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:29.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161131 s, 0.0 kB/s 2024-04-26T20:13:30.014 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:30.062 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:30.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:30.238 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:30.355 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:30.388 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:30.438 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000308031 s, 0.0 kB/s 2024-04-26T20:13:30.489 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:30.538 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:30.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:30.734 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:30.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:30.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:30.980 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219387 s, 0.0 kB/s 2024-04-26T20:13:31.045 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:31.111 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:31.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:31.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:31.372 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:31.413 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:31.463 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023081 s, 0.0 kB/s 2024-04-26T20:13:31.514 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:31.569 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:31.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:31.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:31.854 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:31.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:31.938 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216646 s, 0.0 kB/s 2024-04-26T20:13:32.034 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:32.088 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:32.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:32.289 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:32.402 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:32.467 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:32.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218785 s, 0.0 kB/s 2024-04-26T20:13:32.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:32.613 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:32.764 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:32.882 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:32.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:32.964 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000302096 s, 0.0 kB/s 2024-04-26T20:13:33.024 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:33.072 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:33.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:33.319 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:33.522 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168172 s, 0.0 kB/s 2024-04-26T20:13:33.565 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:33.606 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:33.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:33.731 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:33.873 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:33.906 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166756 s, 0.0 kB/s 2024-04-26T20:13:33.941 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:33.973 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:34.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:34.156 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:34.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:34.336 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:34.407 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000330339 s, 0.0 kB/s 2024-04-26T20:13:34.441 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:34.473 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:34.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:34.600 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:34.705 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:34.748 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:34.782 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152694 s, 0.0 kB/s 2024-04-26T20:13:34.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:34.857 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:34.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:34.973 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:35.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:35.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:35.149 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178065 s, 0.0 kB/s 2024-04-26T20:13:35.183 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:35.277 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:35.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:35.449 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:35.550 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:35.574 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202685 s, 0.0 kB/s 2024-04-26T20:13:35.684 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:35.716 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:35.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:35.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:35.931 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:35.966 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:35.999 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020687 s, 0.0 kB/s 2024-04-26T20:13:36.034 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:36.066 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:36.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:36.225 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:36.360 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:36.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:36.416 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018258 s, 0.0 kB/s 2024-04-26T20:13:36.480 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:36.516 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:36.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:36.650 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:36.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:36.783 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:36.817 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221293 s, 0.0 kB/s 2024-04-26T20:13:36.851 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:36.883 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:36.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:37.008 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:37.105 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:37.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:37.192 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168734 s, 0.0 kB/s 2024-04-26T20:13:37.274 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:37.317 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:37.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:37.526 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:37.676 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:37.726 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163575 s, 0.0 kB/s 2024-04-26T20:13:37.777 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:37.826 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:37.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:37.976 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:38.092 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:38.126 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000284251 s, 0.0 kB/s 2024-04-26T20:13:38.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:38.276 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:38.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:38.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:38.652 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:38.738 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251846 s, 0.0 kB/s 2024-04-26T20:13:38.794 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:38.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:38.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:38.993 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:39.109 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:39.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:39.193 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243588 s, 0.0 kB/s 2024-04-26T20:13:39.340 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:39.394 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:39.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:39.544 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:39.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:39.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159538 s, 0.0 kB/s 2024-04-26T20:13:39.928 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:40.011 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:40.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:40.273 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:40.385 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:40.502 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177751 s, 0.0 kB/s 2024-04-26T20:13:40.553 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:40.611 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:40.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:40.761 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:40.874 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:40.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:40.961 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158715 s, 0.0 kB/s 2024-04-26T20:13:41.012 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:41.061 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:41.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:41.253 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:41.453 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160993 s, 0.0 kB/s 2024-04-26T20:13:41.529 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:41.595 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:41.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:41.747 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:41.862 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:41.895 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:41.945 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016047 s, 0.0 kB/s 2024-04-26T20:13:42.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:42.062 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:42.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:42.245 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:42.363 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:42.403 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:42.453 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243024 s, 0.0 kB/s 2024-04-26T20:13:42.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:42.611 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:42.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:42.754 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:42.869 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:42.937 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:42.987 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172247 s, 0.0 kB/s 2024-04-26T20:13:43.038 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:43.087 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:43.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:43.237 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:43.352 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:43.387 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:43.446 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233121 s, 0.0 kB/s 2024-04-26T20:13:43.497 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:43.565 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:43.712 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:43.857 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:43.887 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:43.938 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190619 s, 0.0 kB/s 2024-04-26T20:13:43.989 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:44.037 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:44.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:44.188 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:44.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:44.338 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:44.388 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000300613 s, 0.0 kB/s 2024-04-26T20:13:44.440 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:44.488 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:44.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:44.702 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:44.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:44.921 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:44.971 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185558 s, 0.0 kB/s 2024-04-26T20:13:45.062 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:45.105 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:45.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:45.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:45.373 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:45.430 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:45.480 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00016117 s, 0.0 kB/s 2024-04-26T20:13:45.531 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:45.643 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:45.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:45.788 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:45.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:45.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:45.988 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195585 s, 0.0 kB/s 2024-04-26T20:13:46.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:46.114 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:46.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:46.272 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:46.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:46.522 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202905 s, 0.0 kB/s 2024-04-26T20:13:46.598 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:46.647 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:46.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:46.797 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:46.911 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:47.022 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203134 s, 0.0 kB/s 2024-04-26T20:13:47.073 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:47.122 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:47.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:47.298 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:47.431 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:47.531 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209129 s, 0.0 kB/s 2024-04-26T20:13:47.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:47.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:47.670 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:13:47.716 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:13:47.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:47.792 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:47.945 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208472 s, 0.0 kB/s 2024-04-26T20:13:48.013 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:48.045 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:48.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:48.229 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:48.329 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:48.362 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:48.396 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219398 s, 0.0 kB/s 2024-04-26T20:13:48.476 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:48.532 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:48.646 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:48.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:48.821 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:48.854 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000252986 s, 0.0 kB/s 2024-04-26T20:13:48.904 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:48.929 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:49.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:49.046 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:49.143 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:49.179 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:49.255 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194925 s, 0.0 kB/s 2024-04-26T20:13:49.297 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:49.346 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:49.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:49.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:49.658 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:49.721 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225655 s, 0.0 kB/s 2024-04-26T20:13:49.756 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:49.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:49.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:49.905 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:50.002 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:50.055 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:50.088 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156732 s, 0.0 kB/s 2024-04-26T20:13:50.123 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:50.189 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:50.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:50.372 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:50.471 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:50.505 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233238 s, 0.0 kB/s 2024-04-26T20:13:50.625 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:50.665 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:50.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:50.789 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:50.923 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:50.956 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000233156 s, 0.0 kB/s 2024-04-26T20:13:51.003 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:51.031 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:51.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:51.148 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:51.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:51.298 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190271 s, 0.0 kB/s 2024-04-26T20:13:51.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:51.515 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:51.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:51.665 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:51.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:51.807 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:51.832 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190287 s, 0.0 kB/s 2024-04-26T20:13:51.866 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:51.898 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:51.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:52.007 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:52.117 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:52.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:52.165 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000214233 s, 0.0 kB/s 2024-04-26T20:13:52.200 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:52.233 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:52.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:52.328 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:52.414 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:52.441 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265928 s, 0.0 kB/s 2024-04-26T20:13:52.534 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:52.558 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:52.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:52.699 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:52.780 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:52.808 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:52.824 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000164637 s, 0.0 kB/s 2024-04-26T20:13:52.850 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:52.892 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:52.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:53.016 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:53.136 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:53.272 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191612 s, 0.0 kB/s 2024-04-26T20:13:53.309 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:53.375 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:53.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:53.558 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:53.656 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:53.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163619 s, 0.0 kB/s 2024-04-26T20:13:53.759 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:53.792 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:53.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:53.950 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:54.048 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:54.084 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:54.117 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165045 s, 0.0 kB/s 2024-04-26T20:13:54.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:54.234 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:54.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:54.366 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:54.484 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:54.518 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:54.551 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258623 s, 0.0 kB/s 2024-04-26T20:13:54.636 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:54.668 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:54.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:54.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:54.900 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:54.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:55.001 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184317 s, 0.0 kB/s 2024-04-26T20:13:55.036 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:55.091 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:55.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:55.379 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:55.410 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:55.451 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173923 s, 0.0 kB/s 2024-04-26T20:13:55.486 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:55.548 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:55.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:55.685 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:55.785 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:55.810 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:55.844 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229933 s, 0.0 kB/s 2024-04-26T20:13:55.878 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:55.911 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:56.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:56.060 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:56.157 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:56.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:56.264 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020503 s, 0.0 kB/s 2024-04-26T20:13:56.303 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:56.336 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:56.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:56.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:56.589 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:56.636 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:56.669 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190857 s, 0.0 kB/s 2024-04-26T20:13:56.703 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:56.736 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:56.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:56.859 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:56.986 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:57.020 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000300672 s, 0.0 kB/s 2024-04-26T20:13:57.054 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:57.086 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:57.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:57.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:57.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:57.411 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:57.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000160868 s, 0.0 kB/s 2024-04-26T20:13:57.540 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:57.581 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:57.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:57.695 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:57.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:57.828 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:57.862 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231532 s, 0.0 kB/s 2024-04-26T20:13:57.896 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:57.931 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:58.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:58.095 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:58.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:58.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225739 s, 0.0 kB/s 2024-04-26T20:13:58.297 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:58.334 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:58.521 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:58.621 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:58.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:58.779 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203975 s, 0.0 kB/s 2024-04-26T20:13:58.814 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:58.846 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:58.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:58.962 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:59.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169653 s, 0.0 kB/s 2024-04-26T20:13:59.164 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:59.233 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:59.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:59.412 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:59.514 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:59.538 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:59.630 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185102 s, 0.0 kB/s 2024-04-26T20:13:59.656 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:13:59.688 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:13:59.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:13:59.805 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:13:59.902 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:13:59.930 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:13:59.963 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212828 s, 0.0 kB/s 2024-04-26T20:14:00.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:00.097 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:00.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:00.238 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:00.389 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:00.422 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:00.455 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223838 s, 0.0 kB/s 2024-04-26T20:14:00.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:00.576 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:00.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:00.730 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:00.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:00.964 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000265105 s, 0.0 kB/s 2024-04-26T20:14:00.999 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:01.031 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:01.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:01.147 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:01.248 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:01.273 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:01.306 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000251498 s, 0.0 kB/s 2024-04-26T20:14:01.375 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:01.456 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:01.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:01.573 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:01.671 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:01.706 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:01.756 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158843 s, 0.0 kB/s 2024-04-26T20:14:01.791 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:01.823 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:01.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:01.949 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:02.049 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:02.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:02.115 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202735 s, 0.0 kB/s 2024-04-26T20:14:02.149 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:02.181 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:02.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:02.302 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:02.399 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:02.465 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000256776 s, 0.0 kB/s 2024-04-26T20:14:02.533 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:02.612 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:02.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:02.732 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:02.899 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153653 s, 0.0 kB/s 2024-04-26T20:14:02.933 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:02.966 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:03.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:03.082 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:03.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:03.241 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:03.274 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213292 s, 0.0 kB/s 2024-04-26T20:14:03.346 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:03.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:03.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:03.503 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:03.596 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:03.633 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:03.667 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000280641 s, 0.0 kB/s 2024-04-26T20:14:03.701 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:03.733 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:03.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:03.867 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:03.962 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025618 s, 0.0 kB/s 2024-04-26T20:14:04.068 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:04.100 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:04.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:04.250 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:04.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:04.426 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:04.476 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159272 s, 0.0 kB/s 2024-04-26T20:14:04.529 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:04.550 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:04.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:04.659 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:04.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:04.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:04.859 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000327497 s, 0.0 kB/s 2024-04-26T20:14:04.877 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:04.892 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:04.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:05.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:05.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:05.211 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:05.234 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187068 s, 0.0 kB/s 2024-04-26T20:14:05.261 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:05.276 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:05.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:05.433 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:05.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:05.593 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0002174 s, 0.0 kB/s 2024-04-26T20:14:05.611 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:05.627 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:05.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:05.727 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:05.910 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205078 s, 0.0 kB/s 2024-04-26T20:14:05.946 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:06.028 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:06.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:06.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:06.312 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:06.344 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:06.411 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274179 s, 0.0 kB/s 2024-04-26T20:14:06.445 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:06.477 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:06.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:06.594 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:06.700 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:06.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:06.769 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00023566 s, 0.0 kB/s 2024-04-26T20:14:06.804 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:06.889 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:06.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:07.039 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:07.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:07.239 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178721 s, 0.0 kB/s 2024-04-26T20:14:07.340 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:07.391 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:07.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:07.564 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:07.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:07.763 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:07.814 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185141 s, 0.0 kB/s 2024-04-26T20:14:07.866 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:07.914 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:08.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:08.064 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:08.179 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:08.231 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:08.298 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208381 s, 0.0 kB/s 2024-04-26T20:14:08.370 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:08.423 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:08.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:08.573 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:08.741 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:08.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:08.823 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260068 s, 0.0 kB/s 2024-04-26T20:14:08.874 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:08.923 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:09.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:09.073 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:09.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:09.223 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:09.273 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224978 s, 0.0 kB/s 2024-04-26T20:14:09.325 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:09.373 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:09.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:09.549 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:09.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:09.698 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:09.749 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00017533 s, 0.0 kB/s 2024-04-26T20:14:09.804 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:09.877 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:09.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:10.024 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:10.140 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:10.174 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158613 s, 0.0 kB/s 2024-04-26T20:14:10.275 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:10.366 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:10.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:10.516 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:10.634 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:10.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:10.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201117 s, 0.0 kB/s 2024-04-26T20:14:10.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:10.927 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:11.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:11.074 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:11.416 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:11.466 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189658 s, 0.0 kB/s 2024-04-26T20:14:11.526 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:11.575 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:11.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:11.725 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:11.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:11.875 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:11.925 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202881 s, 0.0 kB/s 2024-04-26T20:14:11.976 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:12.025 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:12.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:12.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:12.314 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:12.350 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:12.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000148404 s, 0.0 kB/s 2024-04-26T20:14:12.508 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:12.559 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:12.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:12.709 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:12.826 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:12.859 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:12.930 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188196 s, 0.0 kB/s 2024-04-26T20:14:12.977 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:13.026 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:13.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:13.201 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:13.317 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:13.369 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:13.401 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000285728 s, 0.0 kB/s 2024-04-26T20:14:13.452 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:13.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:13.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:13.717 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:13.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:13.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:13.918 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000195998 s, 0.0 kB/s 2024-04-26T20:14:13.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:14.084 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:14.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:14.235 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:14.362 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:14.393 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279322 s, 0.0 kB/s 2024-04-26T20:14:14.469 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:14.530 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:14.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:14.652 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:14.754 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:14.793 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:14.850 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264753 s, 0.0 kB/s 2024-04-26T20:14:14.935 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:14.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:15.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:15.085 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:15.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:15.219 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00032681 s, 0.0 kB/s 2024-04-26T20:14:15.286 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:15.319 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:15.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:15.444 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:15.609 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:15.644 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:15.686 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187243 s, 0.0 kB/s 2024-04-26T20:14:15.728 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:15.769 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:15.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:15.919 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:16.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:16.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:16.120 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196792 s, 0.0 kB/s 2024-04-26T20:14:16.170 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:16.219 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:16.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:16.369 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:16.536 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:16.586 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244245 s, 0.0 kB/s 2024-04-26T20:14:16.637 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:16.686 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:16.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:16.836 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:16.955 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:16.987 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:17.037 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187609 s, 0.0 kB/s 2024-04-26T20:14:17.088 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:17.137 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:17.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:17.335 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:17.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:17.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:17.562 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173571 s, 0.0 kB/s 2024-04-26T20:14:17.613 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:17.662 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:17.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:17.824 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:17.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162089 s, 0.0 kB/s 2024-04-26T20:14:18.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:14:18.180 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:18.229 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:18.280 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:14:18.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:18.435 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:18.558 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:18.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:18.663 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000375487 s, 0.0 kB/s 2024-04-26T20:14:18.714 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:18.763 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:18.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:18.913 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:19.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:19.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192192 s, 0.0 kB/s 2024-04-26T20:14:19.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:19.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:19.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:19.472 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:19.587 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:19.622 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:19.672 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203006 s, 0.0 kB/s 2024-04-26T20:14:19.723 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:19.772 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:19.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:19.964 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:20.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:20.114 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:20.164 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159132 s, 0.0 kB/s 2024-04-26T20:14:20.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:20.264 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:20.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:20.414 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:20.545 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:20.581 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:20.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217849 s, 0.0 kB/s 2024-04-26T20:14:20.682 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:20.756 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:20.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:20.923 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:21.039 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:21.072 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:21.123 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000292251 s, 0.0 kB/s 2024-04-26T20:14:21.174 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:21.223 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:21.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:21.390 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:21.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:21.540 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:21.590 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000159702 s, 0.0 kB/s 2024-04-26T20:14:21.641 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:21.698 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:21.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:21.856 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:21.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:22.031 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:22.081 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171742 s, 0.0 kB/s 2024-04-26T20:14:22.133 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:22.181 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:22.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:22.332 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:22.507 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:22.557 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189798 s, 0.0 kB/s 2024-04-26T20:14:22.672 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:22.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:22.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:22.879 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:23.049 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:23.099 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278509 s, 0.0 kB/s 2024-04-26T20:14:23.150 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:23.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:23.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:23.349 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:23.463 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:23.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:23.605 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152856 s, 0.0 kB/s 2024-04-26T20:14:23.650 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:23.699 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:23.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:23.849 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:23.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:24.099 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221985 s, 0.0 kB/s 2024-04-26T20:14:24.157 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:24.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:24.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:24.358 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:24.522 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:24.558 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:24.608 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172531 s, 0.0 kB/s 2024-04-26T20:14:24.659 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:24.708 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:24.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:24.858 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:24.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:25.008 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:25.058 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000277445 s, 0.0 kB/s 2024-04-26T20:14:25.109 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:25.158 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:25.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:25.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:25.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:25.492 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:25.542 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000157603 s, 0.0 kB/s 2024-04-26T20:14:25.593 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:25.642 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:25.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:25.816 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:25.933 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:25.992 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:26.067 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185207 s, 0.0 kB/s 2024-04-26T20:14:26.119 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:26.241 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:26.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:26.384 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:26.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:26.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:26.639 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213246 s, 0.0 kB/s 2024-04-26T20:14:26.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:26.734 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:26.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:26.884 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:27.001 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:27.034 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:27.084 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168064 s, 0.0 kB/s 2024-04-26T20:14:27.177 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:27.226 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:27.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:27.384 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:27.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000275945 s, 0.0 kB/s 2024-04-26T20:14:27.686 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:27.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:27.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:27.934 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:28.083 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:28.118 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:28.160 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235648 s, 0.0 kB/s 2024-04-26T20:14:28.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:28.243 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:28.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:28.402 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:28.520 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:28.577 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:28.631 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268115 s, 0.0 kB/s 2024-04-26T20:14:28.716 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:28.752 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:28.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:28.919 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:29.038 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:29.086 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:29.136 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163536 s, 0.0 kB/s 2024-04-26T20:14:29.187 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:29.236 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:29.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:29.369 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:29.477 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:29.545 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000216467 s, 0.0 kB/s 2024-04-26T20:14:29.579 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:29.660 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:29.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:29.838 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:29.935 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:29.970 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:30.003 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223507 s, 0.0 kB/s 2024-04-26T20:14:30.063 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:30.121 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:30.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:30.262 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:30.444 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:30.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273423 s, 0.0 kB/s 2024-04-26T20:14:30.572 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:30.612 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:30.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:30.737 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:30.845 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:30.879 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:30.922 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217064 s, 0.0 kB/s 2024-04-26T20:14:30.964 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:31.027 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:31.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:31.212 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:31.356 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000263628 s, 0.0 kB/s 2024-04-26T20:14:31.465 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:31.496 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:31.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:31.613 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:31.712 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:31.796 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:31.830 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227688 s, 0.0 kB/s 2024-04-26T20:14:31.865 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:31.896 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:32.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:32.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:32.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:32.188 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:32.249 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000243189 s, 0.0 kB/s 2024-04-26T20:14:32.313 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:32.347 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:32.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:32.497 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:32.660 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:32.689 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262889 s, 0.0 kB/s 2024-04-26T20:14:32.723 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:32.755 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:32.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:32.893 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:33.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:33.052 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:33.102 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183734 s, 0.0 kB/s 2024-04-26T20:14:33.153 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:33.227 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:33.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:33.394 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:33.508 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:33.544 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:33.594 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169108 s, 0.0 kB/s 2024-04-26T20:14:33.645 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:33.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:33.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:33.844 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:33.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:34.059 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222372 s, 0.0 kB/s 2024-04-26T20:14:34.162 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:34.224 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:34.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:34.369 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:34.486 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:34.519 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:34.578 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228672 s, 0.0 kB/s 2024-04-26T20:14:34.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:34.669 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:34.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:34.845 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:35.005 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:35.053 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162132 s, 0.0 kB/s 2024-04-26T20:14:35.104 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:35.153 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:35.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:35.303 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:35.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:35.562 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165377 s, 0.0 kB/s 2024-04-26T20:14:35.622 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:35.701 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:35.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:35.920 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:36.035 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:36.079 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:36.129 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168594 s, 0.0 kB/s 2024-04-26T20:14:36.180 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:36.230 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:36.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:36.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:36.554 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:36.587 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:36.637 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202632 s, 0.0 kB/s 2024-04-26T20:14:36.730 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:36.787 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:36.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:36.937 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:37.094 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:37.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:37.187 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206752 s, 0.0 kB/s 2024-04-26T20:14:37.239 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:37.288 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:37.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:37.438 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:37.563 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:37.688 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025814 s, 0.0 kB/s 2024-04-26T20:14:37.739 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:37.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:37.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:37.961 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:38.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:38.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:38.182 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000229191 s, 0.0 kB/s 2024-04-26T20:14:38.256 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:38.310 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:38.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:38.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:38.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:38.689 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000279478 s, 0.0 kB/s 2024-04-26T20:14:38.740 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:38.789 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:38.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:38.939 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:39.053 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:39.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:39.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153815 s, 0.0 kB/s 2024-04-26T20:14:39.224 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:39.323 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:39.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:39.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:39.613 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:39.748 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272569 s, 0.0 kB/s 2024-04-26T20:14:39.799 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:39.848 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:39.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:40.030 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:40.262 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:40.315 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000228992 s, 0.0 kB/s 2024-04-26T20:14:40.366 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:40.431 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:40.632 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:40.764 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:40.900 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019412 s, 0.0 kB/s 2024-04-26T20:14:40.967 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:41.032 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:41.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:41.207 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:41.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:41.432 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:41.499 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215762 s, 0.0 kB/s 2024-04-26T20:14:41.566 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:41.632 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:41.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:41.815 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:41.948 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:42.007 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:42.074 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257028 s, 0.0 kB/s 2024-04-26T20:14:42.133 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:42.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:42.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:42.427 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:42.567 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:42.616 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:42.674 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152087 s, 0.0 kB/s 2024-04-26T20:14:42.733 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:42.791 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:42.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:42.975 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:43.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:43.162 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:43.216 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237579 s, 0.0 kB/s 2024-04-26T20:14:43.276 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:43.379 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:43.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:43.575 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:43.690 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:43.725 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:43.792 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186216 s, 0.0 kB/s 2024-04-26T20:14:43.851 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:43.900 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:44.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:44.067 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:44.186 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:44.225 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:44.283 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000192952 s, 0.0 kB/s 2024-04-26T20:14:44.335 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:44.409 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:44.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:44.550 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:44.729 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:44.825 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173284 s, 0.0 kB/s 2024-04-26T20:14:44.877 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:44.926 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:45.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:45.101 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:45.218 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:45.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:45.334 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278636 s, 0.0 kB/s 2024-04-26T20:14:45.385 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:45.435 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:45.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:45.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:45.709 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:45.793 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000314492 s, 0.0 kB/s 2024-04-26T20:14:45.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:45.943 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:46.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:46.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:46.293 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:46.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206592 s, 0.0 kB/s 2024-04-26T20:14:46.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:46.443 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:46.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:46.593 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:46.713 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:46.827 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000359852 s, 0.0 kB/s 2024-04-26T20:14:46.878 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:46.977 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:47.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:47.158 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:47.274 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:47.310 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:47.360 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189999 s, 0.0 kB/s 2024-04-26T20:14:47.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:47.482 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:47.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:47.619 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:47.758 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:47.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:47.853 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000162117 s, 0.0 kB/s 2024-04-26T20:14:47.937 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:47.986 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:48.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:48.169 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:48.285 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:48.319 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187685 s, 0.0 kB/s 2024-04-26T20:14:48.448 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:48.494 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:48.545 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:14:48.666 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:14:48.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:48.824 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:48.978 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:49.020 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:49.070 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000213844 s, 0.0 kB/s 2024-04-26T20:14:49.121 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:49.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:49.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:49.329 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:49.448 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:49.504 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:49.554 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294943 s, 0.0 kB/s 2024-04-26T20:14:49.627 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:49.729 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:49.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:49.879 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:50.061 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206438 s, 0.0 kB/s 2024-04-26T20:14:50.197 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:50.246 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:50.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:50.404 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:50.523 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:50.579 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:50.629 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000361931 s, 0.0 kB/s 2024-04-26T20:14:50.681 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:50.736 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:50.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:50.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:51.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:51.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:51.096 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209261 s, 0.0 kB/s 2024-04-26T20:14:51.161 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:51.205 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:51.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:51.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:51.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:51.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:51.713 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020993 s, 0.0 kB/s 2024-04-26T20:14:51.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:51.813 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:51.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:51.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:52.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:52.097 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:52.180 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000140175 s, 0.0 kB/s 2024-04-26T20:14:52.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:52.272 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:52.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:52.472 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:52.592 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:52.647 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:52.697 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000304864 s, 0.0 kB/s 2024-04-26T20:14:52.748 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:52.797 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:52.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:52.971 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:53.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:53.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000202811 s, 0.0 kB/s 2024-04-26T20:14:53.224 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:53.273 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:53.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:53.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:53.608 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:53.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:53.689 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225373 s, 0.0 kB/s 2024-04-26T20:14:53.741 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:53.798 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:53.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:53.956 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:54.069 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:54.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:54.173 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000238237 s, 0.0 kB/s 2024-04-26T20:14:54.326 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:54.373 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:54.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:54.523 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:54.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:54.727 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:54.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.0001801 s, 0.0 kB/s 2024-04-26T20:14:54.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:54.873 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:55.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:55.032 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:55.172 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:55.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:55.282 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186935 s, 0.0 kB/s 2024-04-26T20:14:55.333 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:55.382 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:55.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:55.532 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:55.648 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:55.691 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:55.741 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200987 s, 0.0 kB/s 2024-04-26T20:14:55.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:55.886 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:55.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:56.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:56.181 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:56.216 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:56.266 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215105 s, 0.0 kB/s 2024-04-26T20:14:56.317 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:56.366 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:56.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:56.524 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:56.640 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:56.674 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:56.724 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166362 s, 0.0 kB/s 2024-04-26T20:14:56.776 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:56.833 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:56.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:56.992 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:57.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:57.175 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:57.242 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232877 s, 0.0 kB/s 2024-04-26T20:14:57.310 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:57.377 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:57.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:57.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:57.677 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:57.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000218976 s, 0.0 kB/s 2024-04-26T20:14:57.876 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:58.000 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:58.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:58.225 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:58.357 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:58.425 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:58.492 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153402 s, 0.0 kB/s 2024-04-26T20:14:58.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:58.659 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:58.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:58.825 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:58.954 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:59.034 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:59.101 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203404 s, 0.0 kB/s 2024-04-26T20:14:59.193 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:59.259 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:59.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:14:59.426 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:14:59.560 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:14:59.626 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:14:59.693 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000310384 s, 0.0 kB/s 2024-04-26T20:14:59.761 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:14:59.826 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:14:59.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:00.110 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:00.243 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:00.309 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:00.376 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208348 s, 0.0 kB/s 2024-04-26T20:15:00.500 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:00.552 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:00.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:00.718 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:00.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:00.927 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188771 s, 0.0 kB/s 2024-04-26T20:15:00.978 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:01.027 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:01.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:01.237 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:01.419 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:01.494 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173821 s, 0.0 kB/s 2024-04-26T20:15:01.545 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:01.602 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:01.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:01.753 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:01.903 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:01.953 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171795 s, 0.0 kB/s 2024-04-26T20:15:02.071 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:02.119 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:02.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:02.328 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:02.443 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:02.486 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000129133 s, 0.0 kB/s 2024-04-26T20:15:02.588 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:02.636 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:02.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:02.836 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:02.951 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:02.987 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:03.093 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150252 s, 0.0 kB/s 2024-04-26T20:15:03.179 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:03.220 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:03.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:03.378 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:03.495 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:03.528 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000247821 s, 0.0 kB/s 2024-04-26T20:15:03.630 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:03.679 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:03.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:03.829 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:03.944 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024801 s, 0.0 kB/s 2024-04-26T20:15:04.113 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:04.162 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:04.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:04.337 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:04.487 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:04.546 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158618 s, 0.0 kB/s 2024-04-26T20:15:04.597 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:04.646 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:04.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:04.838 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:04.957 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:05.046 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029009 s, 0.0 kB/s 2024-04-26T20:15:05.097 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:05.172 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:05.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:05.338 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:05.461 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:05.513 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:05.571 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185629 s, 0.0 kB/s 2024-04-26T20:15:05.656 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:05.705 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:05.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:05.903 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:06.025 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:06.080 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:06.130 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186595 s, 0.0 kB/s 2024-04-26T20:15:06.181 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:06.230 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:06.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:06.388 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:06.558 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:06.663 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:06.714 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000221811 s, 0.0 kB/s 2024-04-26T20:15:06.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:06.841 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:06.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:06.997 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:07.113 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:07.147 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:07.197 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000217667 s, 0.0 kB/s 2024-04-26T20:15:07.248 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:07.297 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:07.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:07.447 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:07.563 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:07.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:07.689 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019812 s, 0.0 kB/s 2024-04-26T20:15:07.769 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:07.814 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:07.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:07.964 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:08.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:08.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:08.215 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000272188 s, 0.0 kB/s 2024-04-26T20:15:08.266 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:08.315 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:08.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:08.576 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:08.723 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:08.773 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030152 s, 0.0 kB/s 2024-04-26T20:15:08.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:08.889 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:09.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:09.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:09.240 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:09.290 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024033 s, 0.0 kB/s 2024-04-26T20:15:09.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:09.391 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:09.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:09.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:09.724 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:09.782 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:09.832 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00022003 s, 0.0 kB/s 2024-04-26T20:15:09.883 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:09.932 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:10.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:10.082 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:10.232 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:10.296 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000175918 s, 0.0 kB/s 2024-04-26T20:15:10.367 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:10.441 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:10.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:10.591 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:10.708 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:10.766 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:10.816 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000143614 s, 0.0 kB/s 2024-04-26T20:15:10.867 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:10.916 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:11.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:11.075 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:11.195 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:11.308 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:11.358 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000206368 s, 0.0 kB/s 2024-04-26T20:15:11.410 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:11.458 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:11.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:11.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:11.792 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:11.834 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:11.902 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203235 s, 0.0 kB/s 2024-04-26T20:15:11.989 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:12.041 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:12.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:12.267 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:12.383 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:12.467 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154627 s, 0.0 kB/s 2024-04-26T20:15:12.519 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:12.567 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:12.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:12.743 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:12.878 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:13.015 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:13.118 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184621 s, 0.0 kB/s 2024-04-26T20:15:13.185 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:13.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:13.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:13.467 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:13.643 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:13.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:13.751 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000235895 s, 0.0 kB/s 2024-04-26T20:15:13.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:13.876 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:14.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:14.035 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:14.150 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:14.210 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:14.268 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227596 s, 0.0 kB/s 2024-04-26T20:15:14.328 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:14.385 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:14.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:14.535 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:14.701 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:14.752 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:14.802 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00025005 s, 0.0 kB/s 2024-04-26T20:15:14.853 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:14.902 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:15.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:15.052 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:15.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:15.202 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:15.258 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196996 s, 0.0 kB/s 2024-04-26T20:15:15.303 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:15.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:15.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:15.544 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:15.736 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:15.777 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:15.827 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215748 s, 0.0 kB/s 2024-04-26T20:15:15.887 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:15.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:16.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:16.111 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:16.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:16.319 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000156047 s, 0.0 kB/s 2024-04-26T20:15:16.404 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:16.482 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:16.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:16.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:16.753 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:16.861 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000412221 s, 0.0 kB/s 2024-04-26T20:15:16.942 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:16.986 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:17.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:17.128 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:17.245 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:17.378 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208905 s, 0.0 kB/s 2024-04-26T20:15:17.429 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:17.478 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:17.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:17.628 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:17.746 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:17.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:17.829 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000226376 s, 0.0 kB/s 2024-04-26T20:15:17.880 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:17.929 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:18.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:18.102 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:18.288 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:18.422 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000267292 s, 0.0 kB/s 2024-04-26T20:15:18.480 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:18.545 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:18.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:18.712 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:18.831 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:18.871 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:18.921 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000169014 s, 0.0 kB/s 2024-04-26T20:15:18.972 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:19.021 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:19.122 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:15:19.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:19.271 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:19.322 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:15:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:19.546 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:19.596 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000223086 s, 0.0 kB/s 2024-04-26T20:15:19.648 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:19.736 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:19.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:19.896 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:20.010 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:20.071 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:20.168 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171487 s, 0.0 kB/s 2024-04-26T20:15:20.215 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:20.263 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:20.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:20.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:20.588 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:20.638 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207469 s, 0.0 kB/s 2024-04-26T20:15:20.749 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:20.788 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:20.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:20.930 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:21.056 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:21.122 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:21.172 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000209982 s, 0.0 kB/s 2024-04-26T20:15:21.223 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:21.272 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:21.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:21.422 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:21.597 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:21.647 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000224032 s, 0.0 kB/s 2024-04-26T20:15:21.699 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:21.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:21.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:21.897 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:22.070 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:22.106 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:22.156 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000222184 s, 0.0 kB/s 2024-04-26T20:15:22.207 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:22.256 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:22.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:22.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:22.520 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:22.556 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:22.606 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188362 s, 0.0 kB/s 2024-04-26T20:15:22.657 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:22.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:22.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:22.881 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:22.998 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:23.040 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:23.092 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018074 s, 0.0 kB/s 2024-04-26T20:15:23.143 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:23.192 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:23.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:23.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:23.491 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:23.592 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000288732 s, 0.0 kB/s 2024-04-26T20:15:23.643 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:23.747 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:23.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:23.926 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:24.043 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:24.101 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:24.151 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257775 s, 0.0 kB/s 2024-04-26T20:15:24.202 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:24.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:24.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:24.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:24.569 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:24.684 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:24.735 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208715 s, 0.0 kB/s 2024-04-26T20:15:24.809 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:24.860 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:24.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:25.018 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:25.137 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:25.193 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:25.243 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231189 s, 0.0 kB/s 2024-04-26T20:15:25.295 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:25.343 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:25.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:25.548 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:25.659 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:25.743 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000181484 s, 0.0 kB/s 2024-04-26T20:15:25.795 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:25.844 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:25.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:26.036 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:26.154 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:26.260 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000166754 s, 0.0 kB/s 2024-04-26T20:15:26.312 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:26.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:26.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:26.561 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:26.678 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:26.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:26.769 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00030575 s, 0.0 kB/s 2024-04-26T20:15:26.820 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:26.932 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:27.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:27.101 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:27.220 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:27.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:27.449 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000245723 s, 0.0 kB/s 2024-04-26T20:15:27.505 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:27.553 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:27.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:27.703 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:27.895 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:27.945 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000200125 s, 0.0 kB/s 2024-04-26T20:15:27.996 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:28.045 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:28.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:28.228 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:28.397 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:28.453 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:28.503 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000241541 s, 0.0 kB/s 2024-04-26T20:15:28.555 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:28.611 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:28.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:28.754 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:28.922 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:28.970 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000262843 s, 0.0 kB/s 2024-04-26T20:15:29.022 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:29.104 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:29.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:29.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:29.371 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:29.404 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:29.454 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273398 s, 0.0 kB/s 2024-04-26T20:15:29.547 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:29.579 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:29.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:29.738 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:29.964 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:30.013 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000231396 s, 0.0 kB/s 2024-04-26T20:15:30.064 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:30.113 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:30.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:30.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:30.353 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:30.380 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:30.413 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220231 s, 0.0 kB/s 2024-04-26T20:15:30.447 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:30.500 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:30.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:30.650 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:30.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:30.855 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261879 s, 0.0 kB/s 2024-04-26T20:15:30.927 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:30.968 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:31.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:31.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:31.196 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:31.230 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:31.264 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00029589 s, 0.0 kB/s 2024-04-26T20:15:31.298 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:31.364 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:31.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:31.480 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:31.580 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:31.605 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:31.685 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000158478 s, 0.0 kB/s 2024-04-26T20:15:31.724 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:31.756 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:31.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:31.898 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:32.058 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:32.098 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220037 s, 0.0 kB/s 2024-04-26T20:15:32.165 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:32.197 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:32.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:32.348 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:32.441 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:32.464 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:32.498 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000150362 s, 0.0 kB/s 2024-04-26T20:15:32.563 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:32.640 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:32.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:32.840 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:32.939 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:32.973 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:33.030 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178713 s, 0.0 kB/s 2024-04-26T20:15:33.077 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:33.115 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:33.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:33.240 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000188112 s, 0.0 kB/s 2024-04-26T20:15:33.475 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:33.507 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:33.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:33.624 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:33.722 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:33.749 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:33.782 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000220344 s, 0.0 kB/s 2024-04-26T20:15:33.817 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:33.849 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:33.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:34.066 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:34.166 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:34.207 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:34.241 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020841 s, 0.0 kB/s 2024-04-26T20:15:34.276 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:34.308 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:34.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:34.424 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:34.558 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000180531 s, 0.0 kB/s 2024-04-26T20:15:34.626 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:34.694 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:34.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:34.832 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:34.933 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:35.047 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:35.075 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000269862 s, 0.0 kB/s 2024-04-26T20:15:35.118 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:35.141 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:35.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:35.258 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:35.359 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:35.458 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:35.492 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000278942 s, 0.0 kB/s 2024-04-26T20:15:35.526 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:35.560 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:35.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:35.675 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:35.773 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:35.925 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258917 s, 0.0 kB/s 2024-04-26T20:15:36.035 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:36.098 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:36.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:36.259 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:36.392 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:36.434 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000152075 s, 0.0 kB/s 2024-04-26T20:15:36.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:36.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:36.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:36.618 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:36.813 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:36.851 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020112 s, 0.0 kB/s 2024-04-26T20:15:36.921 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:36.951 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:37.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:37.068 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:37.194 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:37.235 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:37.268 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000257588 s, 0.0 kB/s 2024-04-26T20:15:37.303 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:37.335 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:37.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:37.469 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:37.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:37.602 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:37.651 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000163265 s, 0.0 kB/s 2024-04-26T20:15:37.687 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:37.719 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:37.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:37.842 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:37.940 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:37.977 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:38.011 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000177731 s, 0.0 kB/s 2024-04-26T20:15:38.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:38.125 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:38.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:38.244 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:38.343 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:38.389 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:38.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197994 s, 0.0 kB/s 2024-04-26T20:15:38.454 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:38.486 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:38.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:38.603 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:38.756 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:38.778 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:38.842 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000189508 s, 0.0 kB/s 2024-04-26T20:15:38.879 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:38.912 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:39.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:39.037 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:39.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:39.170 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:39.203 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000187737 s, 0.0 kB/s 2024-04-26T20:15:39.238 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:39.270 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:39.437 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:39.533 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:39.570 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:39.604 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237838 s, 0.0 kB/s 2024-04-26T20:15:39.638 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:39.670 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:39.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:39.853 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:39.953 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:39.971 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:40.072 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000237944 s, 0.0 kB/s 2024-04-26T20:15:40.155 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:40.196 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:40.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:40.312 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:40.413 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:40.446 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:40.489 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000258172 s, 0.0 kB/s 2024-04-26T20:15:40.522 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:40.554 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:40.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:40.671 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:40.771 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:40.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000204909 s, 0.0 kB/s 2024-04-26T20:15:40.873 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:40.905 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:41.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:41.038 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:41.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:41.238 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00024349 s, 0.0 kB/s 2024-04-26T20:15:41.290 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:41.313 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:41.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:41.422 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:41.555 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000190465 s, 0.0 kB/s 2024-04-26T20:15:41.573 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:41.655 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:41.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:41.756 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:41.836 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:41.881 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000283204 s, 0.0 kB/s 2024-04-26T20:15:41.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:41.972 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:42.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:42.123 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:42.229 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:42.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:42.297 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173055 s, 0.0 kB/s 2024-04-26T20:15:42.332 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:42.364 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:42.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:42.531 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:42.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:42.723 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000185418 s, 0.0 kB/s 2024-04-26T20:15:42.757 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:42.813 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:42.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:42.940 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:43.046 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:43.073 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:43.107 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000168119 s, 0.0 kB/s 2024-04-26T20:15:43.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:43.241 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:43.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:43.373 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:43.474 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:43.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:43.532 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000211593 s, 0.0 kB/s 2024-04-26T20:15:43.608 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:43.640 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:43.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:43.757 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:43.886 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:43.924 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:43.957 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186875 s, 0.0 kB/s 2024-04-26T20:15:44.033 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:44.065 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:44.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:44.182 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:44.281 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:44.316 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:44.349 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000248429 s, 0.0 kB/s 2024-04-26T20:15:44.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:44.416 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:44.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:44.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:44.693 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:44.716 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198769 s, 0.0 kB/s 2024-04-26T20:15:44.816 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:44.856 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:44.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:44.966 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:45.065 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:45.100 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:45.133 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00020707 s, 0.0 kB/s 2024-04-26T20:15:45.167 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:45.200 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:45.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:45.350 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:45.449 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:45.483 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:45.548 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000205744 s, 0.0 kB/s 2024-04-26T20:15:45.584 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:45.617 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:45.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:45.734 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:45.892 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:46.000 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000196086 s, 0.0 kB/s 2024-04-26T20:15:46.035 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:46.067 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:46.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:46.184 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:46.283 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:46.318 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000207329 s, 0.0 kB/s 2024-04-26T20:15:46.402 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:46.434 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:46.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:46.643 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:46.742 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:46.768 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:46.819 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00013914 s, 0.0 kB/s 2024-04-26T20:15:46.870 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:46.901 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:46.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:47.018 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:47.115 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:47.135 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:47.169 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000268172 s, 0.0 kB/s 2024-04-26T20:15:47.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:47.235 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:47.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:47.385 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:47.498 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:47.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000219846 s, 0.0 kB/s 2024-04-26T20:15:47.603 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:47.635 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:47.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:47.769 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:47.866 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:47.919 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:47.952 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000274546 s, 0.0 kB/s 2024-04-26T20:15:47.987 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:48.019 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:48.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:48.152 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:48.255 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:48.286 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:48.319 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227852 s, 0.0 kB/s 2024-04-26T20:15:48.354 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:48.414 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:48.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:48.586 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:48.688 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:48.711 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:48.755 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000294509 s, 0.0 kB/s 2024-04-26T20:15:48.796 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:48.828 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:48.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:48.945 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:49.042 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:49.128 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:49.161 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208919 s, 0.0 kB/s 2024-04-26T20:15:49.196 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:49.228 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:49.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:49.345 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:49.512 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:49.545 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:49.578 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242834 s, 0.0 kB/s 2024-04-26T20:15:49.613 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:49.683 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:49.722 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:15:49.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:15:49.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:49.887 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:50.027 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:50.082 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:50.132 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000182563 s, 0.0 kB/s 2024-04-26T20:15:50.184 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:50.253 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:50.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:50.382 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:50.497 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:50.524 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:50.565 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147305 s, 0.0 kB/s 2024-04-26T20:15:50.609 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:50.674 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:50.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:50.824 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:50.920 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:50.949 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:50.991 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147831 s, 0.0 kB/s 2024-04-26T20:15:51.033 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:51.082 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:51.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:51.208 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:51.316 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:51.358 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:51.399 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179877 s, 0.0 kB/s 2024-04-26T20:15:51.442 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:51.498 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:51.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:51.608 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:51.719 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:51.760 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:51.800 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183981 s, 0.0 kB/s 2024-04-26T20:15:51.876 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:51.908 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:51.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:52.058 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:52.164 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:52.200 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:52.233 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000176487 s, 0.0 kB/s 2024-04-26T20:15:52.268 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:52.300 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:52.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:52.452 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:52.584 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:52.700 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232593 s, 0.0 kB/s 2024-04-26T20:15:52.735 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:52.852 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:52.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:53.009 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:53.110 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:53.142 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:53.176 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000242834 s, 0.0 kB/s 2024-04-26T20:15:53.210 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:53.242 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:53.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:53.401 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:53.502 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:53.534 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:53.568 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183031 s, 0.0 kB/s 2024-04-26T20:15:53.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:53.651 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:53.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:53.768 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:53.877 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:53.901 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:53.985 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000271256 s, 0.0 kB/s 2024-04-26T20:15:54.019 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:54.051 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:54.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:54.177 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:54.278 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:54.318 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:54.368 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000273143 s, 0.0 kB/s 2024-04-26T20:15:54.420 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:54.469 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:54.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:54.619 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:54.769 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:54.802 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:54.852 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000201679 s, 0.0 kB/s 2024-04-26T20:15:54.923 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:54.978 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:55.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:55.127 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:55.246 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:55.302 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:55.352 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000260501 s, 0.0 kB/s 2024-04-26T20:15:55.404 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:55.452 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:55.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:55.619 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:55.733 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:55.769 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:55.843 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000186296 s, 0.0 kB/s 2024-04-26T20:15:55.895 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:55.944 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:56.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:56.094 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:56.210 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:56.244 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:56.295 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000212999 s, 0.0 kB/s 2024-04-26T20:15:56.412 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:56.514 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:56.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:56.661 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:56.779 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:56.820 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:56.870 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000244981 s, 0.0 kB/s 2024-04-26T20:15:56.921 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:56.970 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:57.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:57.170 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:57.284 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:57.438 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261264 s, 0.0 kB/s 2024-04-26T20:15:57.488 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:57.537 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:57.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:57.687 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:57.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:57.837 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255949 s, 0.0 kB/s 2024-04-26T20:15:57.938 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:57.987 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:58.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:58.145 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:58.304 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:58.354 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000183575 s, 0.0 kB/s 2024-04-26T20:15:58.405 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:58.487 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:58.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:58.637 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:58.755 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:58.788 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:58.837 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197238 s, 0.0 kB/s 2024-04-26T20:15:58.889 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:58.938 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:59.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:59.096 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:59.210 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:59.255 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:59.305 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227953 s, 0.0 kB/s 2024-04-26T20:15:59.356 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:59.427 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:15:59.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:15:59.613 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:15:59.732 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:15:59.763 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:15:59.813 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000191554 s, 0.0 kB/s 2024-04-26T20:15:59.865 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:15:59.913 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:00.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:00.064 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:00.314 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000178276 s, 0.0 kB/s 2024-04-26T20:16:00.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:00.414 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:00.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:00.555 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:00.722 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000172759 s, 0.0 kB/s 2024-04-26T20:16:00.824 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:00.884 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:00.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:01.031 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:01.146 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:01.181 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:01.231 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194066 s, 0.0 kB/s 2024-04-26T20:16:01.282 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:01.331 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:01.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:01.489 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:01.606 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:01.639 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000225046 s, 0.0 kB/s 2024-04-26T20:16:01.766 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:01.815 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:01.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:01.965 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:02.095 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:02.131 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:02.181 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000171763 s, 0.0 kB/s 2024-04-26T20:16:02.233 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:02.282 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:02.432 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:02.548 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:02.590 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:02.640 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000194121 s, 0.0 kB/s 2024-04-26T20:16:02.692 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:02.774 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:02.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:02.923 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:03.041 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:03.074 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:03.124 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215032 s, 0.0 kB/s 2024-04-26T20:16:03.177 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:03.224 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:03.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:03.399 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:03.515 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:03.599 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000227039 s, 0.0 kB/s 2024-04-26T20:16:03.650 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:03.724 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:03.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:03.874 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:03.989 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:04.094 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018413 s, 0.0 kB/s 2024-04-26T20:16:04.151 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:04.199 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:04.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:04.349 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:04.465 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:04.499 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:04.549 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000161268 s, 0.0 kB/s 2024-04-26T20:16:04.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:04.649 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:04.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:04.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:04.914 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:04.950 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:05.025 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018006 s, 0.0 kB/s 2024-04-26T20:16:05.076 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:05.125 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:05.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:05.300 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:05.417 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:05.475 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:05.540 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000264072 s, 0.0 kB/s 2024-04-26T20:16:05.601 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:05.650 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:05.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:05.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:05.916 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:05.967 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:06.017 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000261996 s, 0.0 kB/s 2024-04-26T20:16:06.068 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:06.142 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:06.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:06.275 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:06.395 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:06.459 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000255646 s, 0.0 kB/s 2024-04-26T20:16:06.585 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:06.634 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:06.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:06.809 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:06.908 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:06.943 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:06.976 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000197076 s, 0.0 kB/s 2024-04-26T20:16:07.011 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:07.051 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:07.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:07.168 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:07.267 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:07.301 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:07.343 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000198829 s, 0.0 kB/s 2024-04-26T20:16:07.387 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:07.447 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:07.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:07.568 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:07.675 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:07.718 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:07.760 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000203979 s, 0.0 kB/s 2024-04-26T20:16:07.803 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:07.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:07.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:07.968 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:08.070 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:08.110 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000143001 s, 0.0 kB/s 2024-04-26T20:16:08.203 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:08.235 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:08.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:08.369 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:08.503 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:08.535 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:08.569 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00019572 s, 0.0 kB/s 2024-04-26T20:16:08.603 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:08.661 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:08.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:08.811 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:08.929 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:08.961 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00015971 s, 0.0 kB/s 2024-04-26T20:16:09.029 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:09.086 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:09.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:09.203 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:09.386 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:09.419 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000179001 s, 0.0 kB/s 2024-04-26T20:16:09.487 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:09.511 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:09.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:09.636 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:09.767 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:09.795 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:09.829 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000153367 s, 0.0 kB/s 2024-04-26T20:16:09.864 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:09.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:09.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:10.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:10.119 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:10.145 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:10.254 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000147257 s, 0.0 kB/s 2024-04-26T20:16:10.288 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:10.321 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:10.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:10.454 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:10.673 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000154084 s, 0.0 kB/s 2024-04-26T20:16:10.729 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:10.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:10.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:10.911 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:11.003 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:11.029 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:11.063 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000173998 s, 0.0 kB/s 2024-04-26T20:16:11.139 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:11.171 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:11.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:11.296 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:11.399 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:11.438 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:11.490 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000208764 s, 0.0 kB/s 2024-04-26T20:16:11.531 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:11.563 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:11.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:11.680 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:11.783 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:11.805 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:11.841 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000246059 s, 0.0 kB/s 2024-04-26T20:16:11.873 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:11.905 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:12.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:12.055 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:12.156 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:12.189 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000232741 s, 0.0 kB/s 2024-04-26T20:16:12.257 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:12.289 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:12.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:12.422 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:12.564 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:12.621 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000184394 s, 0.0 kB/s 2024-04-26T20:16:12.665 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:12.714 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:12.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:12.864 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:13.023 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:13.073 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000170857 s, 0.0 kB/s 2024-04-26T20:16:13.124 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:13.173 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:13.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:13.323 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:13.439 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:13.481 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:13.523 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000215968 s, 0.0 kB/s 2024-04-26T20:16:13.574 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:13.623 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:13.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:13.840 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:13.956 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:13.992 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:14.040 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.000165411 s, 0.0 kB/s 2024-04-26T20:16:14.183 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:16:14.232 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:16:14.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:16:14.373 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol92860/d216f26c-da4b-4310-9de4-ae7f5b7aaece/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2024-04-26T20:16:14.485 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records in 2024-04-26T20:16:14.524 INFO:teuthology.orchestra.run.smithi088.stderr:0+0 records out 2024-04-26T20:16:14.573 INFO:teuthology.orchestra.run.smithi088.stderr:0 bytes copied, 0.00018389 s, 0.0 kB/s 2024-04-26T20:16:14.624 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol92860 2024-04-26T20:16:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.891+0000 7fad86e00640 1 Processor -- start 2024-04-26T20:16:14.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.892+0000 7fad86e00640 1 -- start start 2024-04-26T20:16:14.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.893+0000 7fad86e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad8006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.893+0000 7fad86e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fad800b6c60 0x7fad801632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:15.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.893+0000 7fad86e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8006cc70 con 0x7fad800b67e0 2024-04-26T20:16:15.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.893+0000 7fad86e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fad80073c80 con 0x7fad800b6c60 2024-04-26T20:16:15.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.893+0000 7fad86e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fad80073f60 con 0x7fad8006c350 2024-04-26T20:16:15.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad85000640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fad800b67e0 0x7fad80162ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52414/0 (socket says 172.21.15.88:52414) 2024-04-26T20:16:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad85000640 1 -- 172.21.15.88:0/996167061 learned_addr learned my addr 172.21.15.88:0/996167061 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:16:15.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad86400640 1 --2- 172.21.15.88:0/996167061 >> v2:172.21.15.88:3300/0 conn(0x7fad800b6c60 0x7fad801632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:15.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad85a00640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad8006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:15.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad7fe00640 1 -- 172.21.15.88:0/996167061 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816975576 0 0) 0x7fad8006cc70 con 0x7fad800b67e0 2024-04-26T20:16:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.894+0000 7fad7fe00640 1 -- 172.21.15.88:0/996167061 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad64003520 con 0x7fad800b67e0 2024-04-26T20:16:15.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.895+0000 7fad86400640 1 -- 172.21.15.88:0/996167061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 msgr2=0x7fad8006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:16:15.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.895+0000 7fad86400640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad8006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:15.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.895+0000 7fad86400640 1 -- 172.21.15.88:0/996167061 >> v1:172.21.15.88:6789/0 conn(0x7fad800b67e0 legacy=0x7fad80162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:15.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.895+0000 7fad86400640 1 -- 172.21.15.88:0/996167061 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad800b4ba0 con 0x7fad800b6c60 2024-04-26T20:16:15.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.896+0000 7fad86400640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b6c60 0x7fad801632b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad78009d90 tx=0x7fad7802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a424fece61816f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:15.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.896+0000 7fad7fe00640 1 -- 172.21.15.88:0/996167061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78007740 con 0x7fad800b6c60 2024-04-26T20:16:15.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.896+0000 7fad7fe00640 1 -- 172.21.15.88:0/996167061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad780078e0 con 0x7fad800b6c60 2024-04-26T20:16:15.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.896+0000 7fad7fe00640 1 -- 172.21.15.88:0/996167061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78007c90 con 0x7fad800b6c60 2024-04-26T20:16:15.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 -- 172.21.15.88:0/996167061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b6c60 msgr2=0x7fad801632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b6c60 0x7fad801632b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fad78009d90 tx=0x7fad7802a660 comp rx=0 tx=0).stop 2024-04-26T20:16:16.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 -- 172.21.15.88:0/996167061 shutdown_connections 2024-04-26T20:16:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b6c60 0x7fad801632b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 --2- 172.21.15.88:0/996167061 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad8006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:16.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 -- 172.21.15.88:0/996167061 >> 172.21.15.88:0/996167061 conn(0x7fad800b38a0 msgr2=0x7fad800b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:16:16.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 -- 172.21.15.88:0/996167061 shutdown_connections 2024-04-26T20:16:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.897+0000 7fad86e00640 1 -- 172.21.15.88:0/996167061 wait complete. 2024-04-26T20:16:16.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.899+0000 7fad86e00640 1 Processor -- start 2024-04-26T20:16:16.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.899+0000 7fad86e00640 1 -- start start 2024-04-26T20:16:16.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad801453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:16.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b67e0 0x7fad800d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:16.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad800b6c60 0x7fad800c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:16.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad800ae8c0 con 0x7fad800b67e0 2024-04-26T20:16:16.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fad80073970 con 0x7fad800b6c60 2024-04-26T20:16:16.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fad800aea40 con 0x7fad8006c350 2024-04-26T20:16:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b67e0 0x7fad800d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:16.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b67e0 0x7fad800d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45916/0 (socket says 172.21.15.88:45916) 2024-04-26T20:16:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 -- 172.21.15.88:0/1918489814 learned_addr learned my addr 172.21.15.88:0/1918489814 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:16:16.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85a00640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad801453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 -- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 msgr2=0x7fad801453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:17.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad86400640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad800b6c60 0x7fad800c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:17.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fad8006c350 0x7fad801453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:17.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 -- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad800b6c60 msgr2=0x7fad800c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:16:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad800b6c60 0x7fad800c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:16:17.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.900+0000 7fad85000640 1 -- 172.21.15.88:0/1918489814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad800d0a80 con 0x7fad800b67e0 2024-04-26T20:16:17.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad85000640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad800b67e0 0x7fad800d0540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c430 tx=0x7fad6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:17.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad800b67e0 2024-04-26T20:16:17.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c005ce0 con 0x7fad800b67e0 2024-04-26T20:16:17.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad86e00640 1 -- 172.21.15.88:0/1918489814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad80132650 con 0x7fad800b67e0 2024-04-26T20:16:17.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0072f0 con 0x7fad800b67e0 2024-04-26T20:16:17.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.901+0000 7fad86e00640 1 -- 172.21.15.88:0/1918489814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad80132c10 con 0x7fad800b67e0 2024-04-26T20:16:17.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.902+0000 7fad86e00640 1 -- 172.21.15.88:0/1918489814 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80069ad0 con 0x7fad800b67e0 2024-04-26T20:16:17.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.903+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fad6c01a070 con 0x7fad800b67e0 2024-04-26T20:16:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.904+0000 7fad7d600640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad6006ca10 0x7fad6006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:16:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.904+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fad6c08b2c0 con 0x7fad800b67e0 2024-04-26T20:16:17.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.904+0000 7fad85a00640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad6006ca10 0x7fad6006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:16:17.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.905+0000 7fad85a00640 1 --2- 172.21.15.88:0/1918489814 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fad6006ca10 0x7fad6006eed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fad80145330 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:16:18.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:14.910+0000 7fad7d600640 1 -- 172.21.15.88:0/1918489814 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad6c059500 con 0x7fad800b67e0 2024-04-26T20:16:18.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:16:15.142+0000 7fad86e00640 1 -- 172.21.15.88:0/1918489814 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol92860", "target": ["mon-mgr", ""]}) -- 0x7fad8006cfc0 con 0x7fad6006ca10 2024-04-26T20:16:20.184 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:16:20.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:16:50.399 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:16:50.442 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:17:20.594 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:17:20.645 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:17:50.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:17:50.788 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:18:14.786 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T20:18:14.787 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:18:14.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:18:14.894 DEBUG:teuthology.orchestra.run.smithi132:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:18:15.008 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:18:15.009 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:18:15.009 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:18:15.009 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:15.067 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:18:15.067 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:15.118 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:15.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.316+0000 7f2baee00640 1 Processor -- start 2024-04-26T20:18:15.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.317+0000 7f2baee00640 1 -- start start 2024-04-26T20:18:15.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2baee00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ba8146680 0x7f2ba80b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2baee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ba80b4b30 0x7f2ba80d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2baee00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba8146a60 con 0x7f2ba80d0e70 2024-04-26T20:18:15.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2baee00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba806e970 con 0x7f2ba8146680 2024-04-26T20:18:15.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2baee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba80d17f0 con 0x7f2ba80b4b30 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2bada00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ba8146680 0x7f2ba80b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2bada00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ba8146680 0x7f2ba80b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42858/0 (socket says 172.21.15.88:42858) 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.318+0000 7f2bada00640 1 -- 172.21.15.88:0/815766318 learned_addr learned my addr 172.21.15.88:0/815766318 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.319+0000 7f2bada00640 1 -- 172.21.15.88:0/815766318 >> v2:172.21.15.132:3300/0 conn(0x7f2ba80b4b30 msgr2=0x7f2ba80d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.319+0000 7f2bada00640 1 --2- 172.21.15.88:0/815766318 >> v2:172.21.15.132:3300/0 conn(0x7f2ba80b4b30 0x7f2ba80d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.319+0000 7f2bada00640 1 -- 172.21.15.88:0/815766318 >> v1:172.21.15.132:6789/0 conn(0x7f2ba80d0e70 legacy=0x7f2ba8160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:15.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.319+0000 7f2bada00640 1 -- 172.21.15.88:0/815766318 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba806dd60 con 0x7f2ba8146680 2024-04-26T20:18:15.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.320+0000 7f2bada00640 1 --2- 172.21.15.88:0/815766318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba8146680 0x7f2ba80b5990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806f370 tx=0x7f2b9c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5758cc3b302edbe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.320+0000 7f2ba7e00640 1 -- 172.21.15.88:0/815766318 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c037070 con 0x7f2ba8146680 2024-04-26T20:18:15.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.320+0000 7f2ba7e00640 1 -- 172.21.15.88:0/815766318 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c032440 con 0x7f2ba8146680 2024-04-26T20:18:15.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.320+0000 7f2ba7e00640 1 -- 172.21.15.88:0/815766318 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c032810 con 0x7f2ba8146680 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 -- 172.21.15.88:0/815766318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba8146680 msgr2=0x7f2ba80b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 --2- 172.21.15.88:0/815766318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba8146680 0x7f2ba80b5990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806f370 tx=0x7f2b9c02a230 comp rx=0 tx=0).stop 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 -- 172.21.15.88:0/815766318 shutdown_connections 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 --2- 172.21.15.88:0/815766318 >> v2:172.21.15.132:3300/0 conn(0x7f2ba80b4b30 0x7f2ba80d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 --2- 172.21.15.88:0/815766318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba8146680 0x7f2ba80b5990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 -- 172.21.15.88:0/815766318 >> 172.21.15.88:0/815766318 conn(0x7f2ba806c740 msgr2=0x7f2ba806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 -- 172.21.15.88:0/815766318 shutdown_connections 2024-04-26T20:18:15.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.321+0000 7f2baee00640 1 -- 172.21.15.88:0/815766318 wait complete. 2024-04-26T20:18:15.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.323+0000 7f2baee00640 1 Processor -- start 2024-04-26T20:18:15.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.323+0000 7f2baee00640 1 -- start start 2024-04-26T20:18:15.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ba80b4b30 0x7f2ba81f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ba8146680 0x7f2ba81f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba806e600 con 0x7f2ba80d0e70 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba806e320 con 0x7f2ba8146680 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2baee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba80cd6a0 con 0x7f2ba80b4b30 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42872/0 (socket says 172.21.15.88:42872) 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 -- 172.21.15.88:0/3849666406 learned_addr learned my addr 172.21.15.88:0/3849666406 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bada00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ba80b4b30 0x7f2ba81f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 -- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ba80b4b30 msgr2=0x7f2ba81f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ba80b4b30 0x7f2ba81f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bad000640 1 -- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ba8146680 msgr2=0x7f2ba81f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.324+0000 7f2bae400640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ba8146680 0x7f2ba81f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2bad000640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ba8146680 0x7f2ba81f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2bad000640 1 -- 172.21.15.88:0/3849666406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba80cb350 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2bad000640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c440 tx=0x7f2b9400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94017070 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b940035f0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940039e0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba80cb630 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.325+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba80cbbf0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.327+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba81f50d0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.327+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f2b9401a070 con 0x7f2ba80d0e70 2024-04-26T20:18:15.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.328+0000 7f2ba5600640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 0x7f2b8806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.328+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f2b9408b120 con 0x7f2ba80d0e70 2024-04-26T20:18:15.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.329+0000 7f2bada00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 0x7f2b8806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.329+0000 7f2bada00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 0x7f2b8806ef00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c034000 tx=0x7f2b9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.333+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b94059370 con 0x7f2ba80d0e70 2024-04-26T20:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.533+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ba80693c0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.536+0000 7f2ba5600640 1 -- 172.21.15.88:0/3849666406 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16754 (secure 0 0 0) 0x7f2b940010c0 con 0x7f2ba80d0e70 2024-04-26T20:18:15.538 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:15.538 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:03:29.112906+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:15.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 msgr2=0x7f2b8806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 0x7f2b8806ef00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c034000 tx=0x7f2b9c02e000 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 msgr2=0x7f2ba80ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c440 tx=0x7f2b9400c900 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 shutdown_connections 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ba8146680 0x7f2ba81f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ba80d0e70 0x7f2ba80ca8b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2b8806ca40 0x7f2b8806ef00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 --2- 172.21.15.88:0/3849666406 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ba80b4b30 0x7f2ba81f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 >> 172.21.15.88:0/3849666406 conn(0x7f2ba806c740 msgr2=0x7f2ba80b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 shutdown_connections 2024-04-26T20:18:15.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.539+0000 7f2baee00640 1 -- 172.21.15.88:0/3849666406 wait complete. 2024-04-26T20:18:15.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:02:41.445900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:42.613083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:18:15.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:18:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.749+0000 7fca90a00640 1 Processor -- start 2024-04-26T20:18:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.750+0000 7fca90a00640 1 -- start start 2024-04-26T20:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.751+0000 7fca90a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 0x7fca8c09ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.751+0000 7fca90a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.751+0000 7fca90a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8c0d2f10 con 0x7fca8c0b1f10 2024-04-26T20:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.751+0000 7fca90a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fca8c0d23e0 con 0x7fca8c162ab0 2024-04-26T20:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.751+0000 7fca90a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c0d1dd0 con 0x7fca8c0b5710 2024-04-26T20:18:15.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.752+0000 7fca81400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.752+0000 7fca81400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:51918/0 (socket says 172.21.15.88:51918) 2024-04-26T20:18:15.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.752+0000 7fca81400640 1 -- 172.21.15.88:0/3287805096 learned_addr learned my addr 172.21.15.88:0/3287805096 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:15.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.752+0000 7fca89e00640 1 --2- 172.21.15.88:0/3287805096 >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 0x7fca8c09ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.753+0000 7fca81400640 1 -- 172.21.15.88:0/3287805096 >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 msgr2=0x7fca8c09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.753+0000 7fca81400640 1 --2- 172.21.15.88:0/3287805096 >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 0x7fca8c09ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.753+0000 7fca81400640 1 -- 172.21.15.88:0/3287805096 >> v1:172.21.15.132:6789/0 conn(0x7fca8c0b1f10 legacy=0x7fca8c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.753+0000 7fca81400640 1 -- 172.21.15.88:0/3287805096 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c162e90 con 0x7fca8c0b5710 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.753+0000 7fca89e00640 1 --2- 172.21.15.88:0/3287805096 >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 0x7fca8c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.754+0000 7fca81400640 1 --2- 172.21.15.88:0/3287805096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca74009870 tx=0x7fca7402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c819bff004aca91e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca89400640 1 -- 172.21.15.88:0/3287805096 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74037070 con 0x7fca8c0b5710 2024-04-26T20:18:15.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca89400640 1 -- 172.21.15.88:0/3287805096 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74032440 con 0x7fca8c0b5710 2024-04-26T20:18:15.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca89400640 1 -- 172.21.15.88:0/3287805096 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74032810 con 0x7fca8c0b5710 2024-04-26T20:18:15.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 -- 172.21.15.88:0/3287805096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 msgr2=0x7fca8c09bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 --2- 172.21.15.88:0/3287805096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca74009870 tx=0x7fca7402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 -- 172.21.15.88:0/3287805096 shutdown_connections 2024-04-26T20:18:15.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 --2- 172.21.15.88:0/3287805096 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c0b5710 0x7fca8c09bf90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 --2- 172.21.15.88:0/3287805096 >> v2:172.21.15.132:3300/0 conn(0x7fca8c162ab0 0x7fca8c09ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.755+0000 7fca90a00640 1 -- 172.21.15.88:0/3287805096 >> 172.21.15.88:0/3287805096 conn(0x7fca8c0b4bf0 msgr2=0x7fca8c0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:15.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.756+0000 7fca90a00640 1 -- 172.21.15.88:0/3287805096 shutdown_connections 2024-04-26T20:18:15.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.756+0000 7fca90a00640 1 -- 172.21.15.88:0/3287805096 wait complete. 2024-04-26T20:18:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.757+0000 7fca90a00640 1 Processor -- start 2024-04-26T20:18:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.757+0000 7fca90a00640 1 -- start start 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b5710 0x7fca8c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 0x7fca8c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c0b25a0 con 0x7fca8c0b1f10 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c0d2070 con 0x7fca8c0b5710 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.758+0000 7fca90a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fca8c150240 con 0x7fca8c162ab0 2024-04-26T20:18:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42874/0 (socket says 172.21.15.88:42874) 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 -- 172.21.15.88:0/642264342 learned_addr learned my addr 172.21.15.88:0/642264342 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca81400640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b5710 0x7fca8c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca8a800640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 0x7fca8c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 -- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 msgr2=0x7fca8c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 0x7fca8c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 -- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b5710 msgr2=0x7fca8c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b5710 0x7fca8c0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 -- 172.21.15.88:0/642264342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c14f630 con 0x7fca8c0b1f10 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca8a800640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 0x7fca8c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.759+0000 7fca89e00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009f60 tx=0x7fca7c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.760+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c018020 con 0x7fca8c0b1f10 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.760+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca7c014680 con 0x7fca8c0b1f10 2024-04-26T20:18:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.760+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c014a50 con 0x7fca8c0b1f10 2024-04-26T20:18:15.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.760+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c14f910 con 0x7fca8c0b1f10 2024-04-26T20:18:15.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.760+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c14fed0 con 0x7fca8c0b1f10 2024-04-26T20:18:15.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.761+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c069650 con 0x7fca8c0b1f10 2024-04-26T20:18:15.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.762+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fca7c005d10 con 0x7fca8c0b1f10 2024-04-26T20:18:15.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.764+0000 7fca82a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 0x7fca6806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:15.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.764+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fca7c08cd80 con 0x7fca8c0b1f10 2024-04-26T20:18:15.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.764+0000 7fca81400640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 0x7fca6806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:15.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.765+0000 7fca81400640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 0x7fca6806eed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca74034000 tx=0x7fca7402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:15.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.768+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca7c022030 con 0x7fca8c0b1f10 2024-04-26T20:18:15.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.969+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca8c0cf550 con 0x7fca8c0b1f10 2024-04-26T20:18:15.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.970+0000 7fca82a00640 1 -- 172.21.15.88:0/642264342 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7fca7c010040 con 0x7fca8c0b1f10 2024-04-26T20:18:15.972 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.972+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 msgr2=0x7fca6806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.972+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 0x7fca6806eed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca74034000 tx=0x7fca7402e000 comp rx=0 tx=0).stop 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 msgr2=0x7fca8c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009f60 tx=0x7fca7c009f90 comp rx=0 tx=0).stop 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 shutdown_connections 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fca6806ca10 0x7fca6806eed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fca8c162ab0 0x7fca8c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca8c0b5710 0x7fca8c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 --2- 172.21.15.88:0/642264342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca8c0b1f10 0x7fca8c0cc7c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:15.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 >> 172.21.15.88:0/642264342 conn(0x7fca8c0b4bf0 msgr2=0x7fca8c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:15.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 shutdown_connections 2024-04-26T20:18:15.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:15.973+0000 7fca90a00640 1 -- 172.21.15.88:0/642264342 wait complete. 2024-04-26T20:18:15.994 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:18:16.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.200+0000 7fd212a00640 1 Processor -- start 2024-04-26T20:18:16.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.201+0000 7fd212a00640 1 -- start start 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd212a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fd20c15f360 0x7fd20c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd212a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20c06f520 con 0x7fd20c0728f0 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd212a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd20c06f800 con 0x7fd20c0b5f20 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd212a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fd20c0b6460 con 0x7fd20c15f360 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd20b400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd20c0728f0 0x7fd20c0b5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:42088/0 (socket says 172.21.15.88:42088) 2024-04-26T20:18:16.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.202+0000 7fd20b400640 1 -- 172.21.15.88:0/765797147 learned_addr learned my addr 172.21.15.88:0/765797147 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:16.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20be00640 1 --2- 172.21.15.88:0/765797147 >> v2:172.21.15.132:3300/0 conn(0x7fd20c15f360 0x7fd20c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15365565 0 0) 0x7fd20c06f520 con 0x7fd20c0728f0 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1f0003520 con 0x7fd20c0728f0 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1620222488 0 0) 0x7fd1f0003520 con 0x7fd20c0728f0 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 >> v2:172.21.15.132:3300/0 conn(0x7fd20c15f360 msgr2=0x7fd20c0702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 --2- 172.21.15.88:0/765797147 >> v2:172.21.15.132:3300/0 conn(0x7fd20c15f360 0x7fd20c0702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 >> v1:172.21.15.132:6789/0 conn(0x7fd20c0b5f20 legacy=0x7fd20c0cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c06fae0 con 0x7fd20c0728f0 2024-04-26T20:18:16.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd1f8001670 con 0x7fd20c0728f0 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.203+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 >> v1:172.21.15.88:6789/0 conn(0x7fd20c0728f0 legacy=0x7fd20c0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1f0003a00 0x7fd1f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 0x7fd1f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0005210 0x7fd1f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c0b6460 con 0x7fd1f0004800 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f000c160 con 0x7fd1f0003a00 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f000c360 con 0x7fd1f0005210 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 0x7fd1f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20be00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1f0003a00 0x7fd1f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 -- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0005210 msgr2=0x7fd1f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0005210 0x7fd1f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 -- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1f0003a00 msgr2=0x7fd1f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1f0003a00 0x7fd1f0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd20b400640 1 -- 172.21.15.88:0/765797147 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f0003520 con 0x7fd1f0004800 2024-04-26T20:18:16.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.204+0000 7fd210a00640 1 -- 172.21.15.88:0/765797147 reap_dead start 2024-04-26T20:18:16.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.205+0000 7fd20b400640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 0x7fd1f0004c80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d6895e3651db9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8007c20 con 0x7fd1f0004800 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f8005af0 con 0x7fd1f0004800 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd20aa00640 1 -- 172.21.15.88:0/765797147 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8007070 con 0x7fd1f0004800 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 -- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 msgr2=0x7fd1f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 0x7fd1f0004c80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f0005830 comp rx=0 tx=0).stop 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 -- 172.21.15.88:0/765797147 shutdown_connections 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 --2- 172.21.15.88:0/765797147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f0004800 0x7fd1f0004c80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 -- 172.21.15.88:0/765797147 >> 172.21.15.88:0/765797147 conn(0x7fd20c070b60 msgr2=0x7fd20c06c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:16.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 -- 172.21.15.88:0/765797147 shutdown_connections 2024-04-26T20:18:16.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.206+0000 7fd212a00640 1 -- 172.21.15.88:0/765797147 wait complete. 2024-04-26T20:18:16.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.208+0000 7fd212a00640 1 Processor -- start 2024-04-26T20:18:16.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.208+0000 7fd212a00640 1 -- start start 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 0x7fd20c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd20c157c60 0x7fd20c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c0702e0 con 0x7fd20c14f040 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c073be0 con 0x7fd20c157c60 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd212a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd20c070460 con 0x7fd1f0004800 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd20b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd20b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42892/0 (socket says 172.21.15.88:42892) 2024-04-26T20:18:16.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.209+0000 7fd20b400640 1 -- 172.21.15.88:0/514036510 learned_addr learned my addr 172.21.15.88:0/514036510 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20be00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 0x7fd20c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd210a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd20c157c60 0x7fd20c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 -- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 msgr2=0x7fd20c14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 0x7fd20c14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 -- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd20c157c60 msgr2=0x7fd20c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd20c157c60 0x7fd20c15f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 -- 172.21.15.88:0/514036510 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c1f5ee0 con 0x7fd20c14f040 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20be00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 0x7fd20c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd20b400640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c1d0 tx=0x7fd20c06fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8040020 con 0x7fd20c14f040 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f80047e0 con 0x7fd20c14f040 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.210+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8037040 con 0x7fd20c14f040 2024-04-26T20:18:16.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.211+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c156c40 con 0x7fd20c14f040 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.211+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c15fe90 con 0x7fd20c14f040 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.212+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c069650 con 0x7fd20c14f040 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.213+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fd1f8043020 con 0x7fd20c14f040 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.214+0000 7fd203e00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 0x7fd1ec06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.214+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fd1f80abad0 con 0x7fd20c14f040 2024-04-26T20:18:16.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.214+0000 7fd20be00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 0x7fd1ec06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.215+0000 7fd20be00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 0x7fd1ec06eed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd20c153200 tx=0x7fd1fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.218+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd1f804a030 con 0x7fd20c14f040 2024-04-26T20:18:16.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.415+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd20c15eaa0 con 0x7fd20c14f040 2024-04-26T20:18:16.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.416+0000 7fd203e00640 1 -- 172.21.15.88:0/514036510 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7fd1f8079d20 con 0x7fd20c14f040 2024-04-26T20:18:16.417 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:18:16.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 msgr2=0x7fd1ec06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 0x7fd1ec06eed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd20c153200 tx=0x7fd1fc008040 comp rx=0 tx=0).stop 2024-04-26T20:18:16.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 msgr2=0x7fd20c157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800c1d0 tx=0x7fd20c06fcb0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 shutdown_connections 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd20c157c60 0x7fd20c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd20c14f040 0x7fd20c157720 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.418+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd1ec06ca10 0x7fd1ec06eed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.419+0000 7fd212a00640 1 --2- 172.21.15.88:0/514036510 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd1f0004800 0x7fd20c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.419+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 >> 172.21.15.88:0/514036510 conn(0x7fd20c070b60 msgr2=0x7fd20c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.419+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 shutdown_connections 2024-04-26T20:18:16.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.419+0000 7fd212a00640 1 -- 172.21.15.88:0/514036510 wait complete. 2024-04-26T20:18:16.439 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:18:16.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.645+0000 7f3377200640 1 Processor -- start 2024-04-26T20:18:16.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.645+0000 7f3377200640 1 -- start start 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.646+0000 7f3377200640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f33780b5ca0 0x7f33780d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.646+0000 7f3377200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f33780d12a0 0x7f33780b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.646+0000 7f3377200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3378160b20 con 0x7f33781467a0 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f3377200640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3378146b80 con 0x7f33780b5ca0 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f3377200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3378161730 con 0x7f33780d12a0 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f33780b5ca0 0x7f33780d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f33780b5ca0 0x7f33780d0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42898/0 (socket says 172.21.15.88:42898) 2024-04-26T20:18:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 -- 172.21.15.88:0/3248814265 learned_addr learned my addr 172.21.15.88:0/3248814265 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f3376800640 1 --2- 172.21.15.88:0/3248814265 >> v2:172.21.15.132:3300/0 conn(0x7f33780d12a0 0x7f33780b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 -- 172.21.15.88:0/3248814265 >> v2:172.21.15.132:3300/0 conn(0x7f33780d12a0 msgr2=0x7f33780b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 --2- 172.21.15.88:0/3248814265 >> v2:172.21.15.132:3300/0 conn(0x7f33780d12a0 0x7f33780b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 -- 172.21.15.88:0/3248814265 >> v1:172.21.15.132:6789/0 conn(0x7f33781467a0 legacy=0x7f337815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.647+0000 7f336fe00640 1 -- 172.21.15.88:0/3248814265 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3378160e00 con 0x7f33780b5ca0 2024-04-26T20:18:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.649+0000 7f336fe00640 1 --2- 172.21.15.88:0/3248814265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780b5ca0 0x7f33780d0d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33780b6130 tx=0x7f336002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3585da90f7b5301 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.649+0000 7f3375400640 1 -- 172.21.15.88:0/3248814265 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360037070 con 0x7f33780b5ca0 2024-04-26T20:18:16.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.649+0000 7f3375400640 1 -- 172.21.15.88:0/3248814265 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3360032440 con 0x7f33780b5ca0 2024-04-26T20:18:16.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.649+0000 7f3375400640 1 -- 172.21.15.88:0/3248814265 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360032810 con 0x7f33780b5ca0 2024-04-26T20:18:16.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 -- 172.21.15.88:0/3248814265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780b5ca0 msgr2=0x7f33780d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 --2- 172.21.15.88:0/3248814265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780b5ca0 0x7f33780d0d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f33780b6130 tx=0x7f336002a230 comp rx=0 tx=0).stop 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 -- 172.21.15.88:0/3248814265 shutdown_connections 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 --2- 172.21.15.88:0/3248814265 >> v2:172.21.15.132:3300/0 conn(0x7f33780d12a0 0x7f33780b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 --2- 172.21.15.88:0/3248814265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780b5ca0 0x7f33780d0d60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 -- 172.21.15.88:0/3248814265 >> 172.21.15.88:0/3248814265 conn(0x7f337806c530 msgr2=0x7f337806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 -- 172.21.15.88:0/3248814265 shutdown_connections 2024-04-26T20:18:16.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.650+0000 7f3377200640 1 -- 172.21.15.88:0/3248814265 wait complete. 2024-04-26T20:18:16.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.652+0000 7f3377200640 1 Processor -- start 2024-04-26T20:18:16.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.652+0000 7f3377200640 1 -- start start 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 0x7f33781417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33781467a0 0x7f33780c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33781613c0 con 0x7f33780d12a0 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f33780d16d0 con 0x7f33781467a0 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3377200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f33780cf100 con 0x7f33780b5ca0 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42900/0 (socket says 172.21.15.88:42900) 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 -- 172.21.15.88:0/3032896498 learned_addr learned my addr 172.21.15.88:0/3032896498 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3375e00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 0x7f33781417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 -- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 msgr2=0x7f33781417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f3376800640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33781467a0 0x7f33780c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 0x7f33781417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 -- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33781467a0 msgr2=0x7f33780c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33781467a0 0x7f33780c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.653+0000 7f336fe00640 1 -- 172.21.15.88:0/3032896498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780c5e90 con 0x7f33780d12a0 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f3375e00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 0x7f33781417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:16.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f336fe00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f336002a270 tx=0x7f33600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360041020 con 0x7f33780d12a0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33600306c0 con 0x7f33780d12a0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360037040 con 0x7f33780d12a0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33780c6170 con 0x7f33780d12a0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.654+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33780c6730 con 0x7f33780d12a0 2024-04-26T20:18:16.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.655+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33780693c0 con 0x7f33780d12a0 2024-04-26T20:18:16.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.656+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f3360044020 con 0x7f33780d12a0 2024-04-26T20:18:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.657+0000 7f336e000640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 0x7f335406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.657+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f33600ac200 con 0x7f33780d12a0 2024-04-26T20:18:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.658+0000 7f3375e00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 0x7f335406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.658+0000 7f3375e00640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 0x7f335406ef00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f336800a830 tx=0x7f3368008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:16.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.662+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f336004b070 con 0x7f33780d12a0 2024-04-26T20:18:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.861+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3378141f10 con 0x7f33780d12a0 2024-04-26T20:18:16.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.862+0000 7f336e000640 1 -- 172.21.15.88:0/3032896498 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7f336007a4d0 con 0x7f33780d12a0 2024-04-26T20:18:16.863 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:18:16.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 msgr2=0x7f335406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 0x7f335406ef00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f336800a830 tx=0x7f3368008040 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 msgr2=0x7f33780c6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f336002a270 tx=0x7f33600046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 shutdown_connections 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33781467a0 0x7f33780c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33780d12a0 0x7f33780c6f30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f335406ca40 0x7f335406ef00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 --2- 172.21.15.88:0/3032896498 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f33780b5ca0 0x7f33781417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 >> 172.21.15.88:0/3032896498 conn(0x7f337806c530 msgr2=0x7f33780b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 shutdown_connections 2024-04-26T20:18:16.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:16.865+0000 7f3377200640 1 -- 172.21.15.88:0/3032896498 wait complete. 2024-04-26T20:18:16.886 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:18:17.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.084+0000 7f3afae00640 1 Processor -- start 2024-04-26T20:18:17.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.084+0000 7f3afae00640 1 -- start start 2024-04-26T20:18:17.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3afae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3af40b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3afae00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3af415cff0 0x7f3af4163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3afae00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af4163d40 con 0x7f3af40aed90 2024-04-26T20:18:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3afae00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3af40afba0 con 0x7f3af415cff0 2024-04-26T20:18:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3afae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3af4163020 con 0x7f3af415cc10 2024-04-26T20:18:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3af8e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3af40aed90 0x7f3af406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:42098/0 (socket says 172.21.15.88:42098) 2024-04-26T20:18:17.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3af8e00640 1 -- 172.21.15.88:0/3104353201 learned_addr learned my addr 172.21.15.88:0/3104353201 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:17.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.086+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332675726 0 0) 0x7f3af4163d40 con 0x7f3af40aed90 2024-04-26T20:18:17.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ad4003520 con 0x7f3af40aed90 2024-04-26T20:18:17.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3af40b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af3400640 1 --2- 172.21.15.88:0/3104353201 >> v2:172.21.15.132:3300/0 conn(0x7f3af415cff0 0x7f3af4163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3614815608 0 0) 0x7f3ad4003520 con 0x7f3af40aed90 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 msgr2=0x7f3af40b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3af40b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 >> v2:172.21.15.132:3300/0 conn(0x7f3af415cff0 msgr2=0x7f3af4163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 --2- 172.21.15.88:0/3104353201 >> v2:172.21.15.132:3300/0 conn(0x7f3af415cff0 0x7f3af4163800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af40af5e0 con 0x7f3af40aed90 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f3aec001670 con 0x7f3af40aed90 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3af40b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 >> v1:172.21.15.88:6789/0 conn(0x7f3af40aed90 legacy=0x7f3af406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3ad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40afba0 con 0x7f3ad40039b0 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3af4163020 con 0x7f3ad4004800 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad400ba40 con 0x7f3af415cc10 2024-04-26T20:18:17.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.087+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af8e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3ad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3e00640 1 -- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 msgr2=0x7f3ad40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3af415cc10 0x7f3ad40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3e00640 1 -- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad4004800 msgr2=0x7f3ad4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3400640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3e00640 1 -- 172.21.15.88:0/3104353201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad4003520 con 0x7f3ad40039b0 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.088+0000 7f3af3400640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:17.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.089+0000 7f3af3e00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40098a0 tx=0x7f3ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cddd170fa86c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.089+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae400f040 con 0x7f3ad40039b0 2024-04-26T20:18:17.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.089+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae402ace0 con 0x7f3ad40039b0 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.089+0000 7f3af2a00640 1 -- 172.21.15.88:0/3104353201 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4004250 con 0x7f3ad40039b0 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 -- 172.21.15.88:0/3104353201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ad40039b0 msgr2=0x7f3ad4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 --2- 172.21.15.88:0/3104353201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40098a0 tx=0x7f3ad4005830 comp rx=0 tx=0).stop 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3af8e00640 1 -- 172.21.15.88:0/3104353201 reap_dead start 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 -- 172.21.15.88:0/3104353201 shutdown_connections 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 -- 172.21.15.88:0/3104353201 >> 172.21.15.88:0/3104353201 conn(0x7f3af40b4490 msgr2=0x7f3af40d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 -- 172.21.15.88:0/3104353201 shutdown_connections 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.090+0000 7f3afae00640 1 -- 172.21.15.88:0/3104353201 wait complete. 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.092+0000 7f3afae00640 1 Processor -- start 2024-04-26T20:18:17.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.092+0000 7f3afae00640 1 -- start start 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad40039b0 0x7f3af415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40d1190 0x7f3af4161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3af40af200 con 0x7f3af41462f0 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3af41632e0 con 0x7f3af40d1190 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3afae00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3af40af380 con 0x7f3ad40039b0 2024-04-26T20:18:17.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3af8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42912/0 (socket says 172.21.15.88:42912) 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3af8e00640 1 -- 172.21.15.88:0/1296962374 learned_addr learned my addr 172.21.15.88:0/1296962374 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.093+0000 7f3af3e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad40039b0 0x7f3af415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 -- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad40039b0 msgr2=0x7f3af415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad40039b0 0x7f3af415d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 -- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40d1190 msgr2=0x7f3af4161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40d1190 0x7f3af4161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 -- 172.21.15.88:0/1296962374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af4161850 con 0x7f3af41462f0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3af8e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3af40b1550 tx=0x7f3aec00d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec007440 con 0x7f3af41462f0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aec0075e0 con 0x7f3af41462f0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af40d15f0 con 0x7f3af41462f0 2024-04-26T20:18:17.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004a20 con 0x7f3af41462f0 2024-04-26T20:18:17.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.094+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af4161dc0 con 0x7f3af41462f0 2024-04-26T20:18:17.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.096+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af4069540 con 0x7f3af41462f0 2024-04-26T20:18:17.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.096+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f3aec01b020 con 0x7f3af41462f0 2024-04-26T20:18:17.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.097+0000 7f3aebe00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 0x7f3ad006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.097+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f3aec08b7c0 con 0x7f3af41462f0 2024-04-26T20:18:17.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.098+0000 7f3af3e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 0x7f3ad006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.098+0000 7f3af3e00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 0x7f3ad006eed0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40098a0 tx=0x7f3ae402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.102+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3aec059a50 con 0x7f3af41462f0 2024-04-26T20:18:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.320+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3af40b22e0 con 0x7f3af41462f0 2024-04-26T20:18:17.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.321+0000 7f3aebe00640 1 -- 172.21.15.88:0/1296962374 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7f3aec059350 con 0x7f3af41462f0 2024-04-26T20:18:17.322 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:18:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.323+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 msgr2=0x7f3ad006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.323+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 0x7f3ad006eed0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40098a0 tx=0x7f3ae402e000 comp rx=0 tx=0).stop 2024-04-26T20:18:17.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.323+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 msgr2=0x7f3af4146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.323+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3af40b1550 tx=0x7f3aec00d950 comp rx=0 tx=0).stop 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 shutdown_connections 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3af41462f0 0x7f3af4146770 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af40d1190 0x7f3af4161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3ad006ca10 0x7f3ad006eed0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 --2- 172.21.15.88:0/1296962374 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3ad40039b0 0x7f3af415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 >> 172.21.15.88:0/1296962374 conn(0x7f3af40b4490 msgr2=0x7f3af4157490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 shutdown_connections 2024-04-26T20:18:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.324+0000 7f3afae00640 1 -- 172.21.15.88:0/1296962374 wait complete. 2024-04-26T20:18:17.344 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-04-26T20:18:17.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:17.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:18:17.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[53646]: fuse finished with error 0 and tester_r 0 2024-04-26T20:18:17.438 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:18:17.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:17.438 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:18:17.610 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:18:17.610 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:18:17.610 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:17.668 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:18:17.669 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:17.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:17.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.924+0000 7f04b1000640 1 Processor -- start 2024-04-26T20:18:17.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.924+0000 7f04b1000640 1 -- start start 2024-04-26T20:18:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04b1000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f04ac0b58a0 0x7f04ac0d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04b1000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04ac070670 con 0x7f04ac06f230 2024-04-26T20:18:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04b1000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04ac070950 con 0x7f04ac09ba50 2024-04-26T20:18:17.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04b1000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f04ac070420 con 0x7f04ac0b58a0 2024-04-26T20:18:17.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04aa800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f04ac06f230 0x7f04ac0d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:42114/0 (socket says 172.21.15.88:42114) 2024-04-26T20:18:17.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04aa800640 1 -- 172.21.15.88:0/3985421874 learned_addr learned my addr 172.21.15.88:0/3985421874 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:17.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.926+0000 7f04a9400640 1 --2- 172.21.15.88:0/3985421874 >> v2:172.21.15.88:3300/0 conn(0x7f04ac0b58a0 0x7f04ac0d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.927+0000 7f04a8a00640 1 -- 172.21.15.88:0/3985421874 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2192822462 0 0) 0x7f04ac070670 con 0x7f04ac06f230 2024-04-26T20:18:17.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.927+0000 7f04a8a00640 1 -- 172.21.15.88:0/3985421874 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0490003520 con 0x7f04ac06f230 2024-04-26T20:18:17.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.927+0000 7f04a9400640 1 -- 172.21.15.88:0/3985421874 >> v1:172.21.15.132:6789/0 conn(0x7f04ac09ba50 legacy=0x7f04ac09be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:17.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.927+0000 7f04a9400640 1 -- 172.21.15.88:0/3985421874 >> v1:172.21.15.88:6789/0 conn(0x7f04ac06f230 legacy=0x7f04ac0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.927+0000 7f04a9400640 1 -- 172.21.15.88:0/3985421874 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac06e9e0 con 0x7f04ac0b58a0 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04a9400640 1 --2- 172.21.15.88:0/3985421874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac0b58a0 0x7f04ac0d12d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f049802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411f0b34b5b66412 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04a8a00640 1 -- 172.21.15.88:0/3985421874 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498037070 con 0x7f04ac0b58a0 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04a8a00640 1 -- 172.21.15.88:0/3985421874 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049802ae50 con 0x7f04ac0b58a0 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04a8a00640 1 -- 172.21.15.88:0/3985421874 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498032660 con 0x7f04ac0b58a0 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04b1000640 1 -- 172.21.15.88:0/3985421874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac0b58a0 msgr2=0x7f04ac0d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.928+0000 7f04b1000640 1 --2- 172.21.15.88:0/3985421874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac0b58a0 0x7f04ac0d12d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f04980097a0 tx=0x7f049802a140 comp rx=0 tx=0).stop 2024-04-26T20:18:17.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.929+0000 7f04b1000640 1 -- 172.21.15.88:0/3985421874 shutdown_connections 2024-04-26T20:18:17.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.929+0000 7f04b1000640 1 --2- 172.21.15.88:0/3985421874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac0b58a0 0x7f04ac0d12d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.929+0000 7f04b1000640 1 -- 172.21.15.88:0/3985421874 >> 172.21.15.88:0/3985421874 conn(0x7f04ac0b4b70 msgr2=0x7f04ac0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:17.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.929+0000 7f04b1000640 1 -- 172.21.15.88:0/3985421874 shutdown_connections 2024-04-26T20:18:17.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.929+0000 7f04b1000640 1 -- 172.21.15.88:0/3985421874 wait complete. 2024-04-26T20:18:17.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.931+0000 7f04b1000640 1 Processor -- start 2024-04-26T20:18:17.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.931+0000 7f04b1000640 1 -- start start 2024-04-26T20:18:17.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ac06f230 0x7f04ac0b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ac0b58a0 0x7f04ac1f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac0cf440 con 0x7f04ac09ba50 2024-04-26T20:18:17.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac06e790 con 0x7f04ac0b58a0 2024-04-26T20:18:17.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04b1000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac0d2d00 con 0x7f04ac06f230 2024-04-26T20:18:17.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42932/0 (socket says 172.21.15.88:42932) 2024-04-26T20:18:17.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 -- 172.21.15.88:0/399079275 learned_addr learned my addr 172.21.15.88:0/399079275 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 -- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ac06f230 msgr2=0x7f04ac0b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ac06f230 0x7f04ac0b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 -- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ac0b58a0 msgr2=0x7f04ac1f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ac0b58a0 0x7f04ac1f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.932+0000 7f04a9400640 1 -- 172.21.15.88:0/399079275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac0b6e60 con 0x7f04ac09ba50 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04a9400640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0498035040 tx=0x7f04980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498042070 con 0x7f04ac09ba50 2024-04-26T20:18:17.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049803d440 con 0x7f04ac09ba50 2024-04-26T20:18:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498037040 con 0x7f04ac09ba50 2024-04-26T20:18:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac1f59d0 con 0x7f04ac09ba50 2024-04-26T20:18:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.933+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac0b6120 con 0x7f04ac09ba50 2024-04-26T20:18:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.934+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac069540 con 0x7f04ac09ba50 2024-04-26T20:18:17.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.935+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f0498045070 con 0x7f04ac09ba50 2024-04-26T20:18:17.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.936+0000 7f04a2000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 0x7f048c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:17.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.936+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f04980ac510 con 0x7f04ac09ba50 2024-04-26T20:18:17.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.936+0000 7f04a9e00640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 0x7f048c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:17.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.937+0000 7f04a9e00640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 0x7f048c06eed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f049c003cc0 tx=0x7f049c0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:17.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:17.940+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f049807a7e0 con 0x7f04ac09ba50 2024-04-26T20:18:18.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.142+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04ac0b3930 con 0x7f04ac09ba50 2024-04-26T20:18:18.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.144+0000 7f04a2000640 1 -- 172.21.15.88:0/399079275 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16754 (secure 0 0 0) 0x7f049807a0e0 con 0x7f04ac09ba50 2024-04-26T20:18:18.146 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:18.146 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:03:29.112906+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 msgr2=0x7f048c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 0x7f048c06eed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f049c003cc0 tx=0x7f049c0079b0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 msgr2=0x7f04ac0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f0498035040 tx=0x7f04980046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 shutdown_connections 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04ac0b58a0 0x7f04ac1f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04ac09ba50 0x7f04ac0c7e70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f048c06ca10 0x7f048c06eed0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 --2- 172.21.15.88:0/399079275 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f04ac06f230 0x7f04ac0b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.146+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 >> 172.21.15.88:0/399079275 conn(0x7f04ac0b4b70 msgr2=0x7f04ac0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.147+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 shutdown_connections 2024-04-26T20:18:18.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.147+0000 7f04b1000640 1 -- 172.21.15.88:0/399079275 wait complete. 2024-04-26T20:18:18.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:02:41.445900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:42.613083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:18:18.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:18:18.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.370+0000 7f0326400640 1 Processor -- start 2024-04-26T20:18:18.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.371+0000 7f0326400640 1 -- start start 2024-04-26T20:18:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.372+0000 7f0326400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0320147f20 0x7f03200b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.372+0000 7f0326400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0320073da0 0x7f032006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.372+0000 7f0326400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03200afed0 con 0x7f032006ccb0 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.372+0000 7f0326400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f032006d090 con 0x7f0320147f20 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.372+0000 7f0326400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f03200ade20 con 0x7f0320073da0 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f032006ccb0 0x7f0320072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:42126/0 (socket says 172.21.15.88:42126) 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031fe00640 1 -- 172.21.15.88:0/3403277115 learned_addr learned my addr 172.21.15.88:0/3403277115 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031f400640 1 --2- 172.21.15.88:0/3403277115 >> v2:172.21.15.88:3300/0 conn(0x7f0320147f20 0x7f03200b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f0317e00640 1 --2- 172.21.15.88:0/3403277115 >> v2:172.21.15.132:3300/0 conn(0x7f0320073da0 0x7f032006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031f400640 1 -- 172.21.15.88:0/3403277115 >> v2:172.21.15.132:3300/0 conn(0x7f0320073da0 msgr2=0x7f032006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031f400640 1 --2- 172.21.15.88:0/3403277115 >> v2:172.21.15.132:3300/0 conn(0x7f0320073da0 0x7f032006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031f400640 1 -- 172.21.15.88:0/3403277115 >> v1:172.21.15.88:6789/0 conn(0x7f032006ccb0 legacy=0x7f0320072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031f400640 1 -- 172.21.15.88:0/3403277115 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03200ae100 con 0x7f0320147f20 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.373+0000 7f031ea00640 1 -- 172.21.15.88:0/3403277115 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006813612 0 0) 0x7f0318001490 con 0x7f032006ccb0 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.374+0000 7f031f400640 1 --2- 172.21.15.88:0/3403277115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f03200b5c20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03200732d0 tx=0x7f031002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf69c9ef170ff19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f031ea00640 1 -- 172.21.15.88:0/3403277115 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310037070 con 0x7f0320147f20 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f031ea00640 1 -- 172.21.15.88:0/3403277115 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0310032440 con 0x7f0320147f20 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f031ea00640 1 -- 172.21.15.88:0/3403277115 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310032810 con 0x7f0320147f20 2024-04-26T20:18:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 -- 172.21.15.88:0/3403277115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 msgr2=0x7f03200b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 --2- 172.21.15.88:0/3403277115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f03200b5c20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03200732d0 tx=0x7f031002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 -- 172.21.15.88:0/3403277115 shutdown_connections 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 --2- 172.21.15.88:0/3403277115 >> v2:172.21.15.132:3300/0 conn(0x7f0320073da0 0x7f032006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 --2- 172.21.15.88:0/3403277115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f03200b5c20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 -- 172.21.15.88:0/3403277115 >> 172.21.15.88:0/3403277115 conn(0x7f03200b2d50 msgr2=0x7f03200b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.375+0000 7f0326400640 1 -- 172.21.15.88:0/3403277115 shutdown_connections 2024-04-26T20:18:18.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.376+0000 7f0326400640 1 -- 172.21.15.88:0/3403277115 wait complete. 2024-04-26T20:18:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.377+0000 7f0326400640 1 Processor -- start 2024-04-26T20:18:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.377+0000 7f0326400640 1 -- start start 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f032006ccb0 0x7f032006f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0320073da0 0x7f032015e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03200ae9a0 con 0x7f0320147f20 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f03200ae6c0 con 0x7f032006ccb0 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f0326400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0320074230 con 0x7f0320073da0 2024-04-26T20:18:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.378+0000 7f031fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42936/0 (socket says 172.21.15.88:42936) 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 -- 172.21.15.88:0/1095225050 learned_addr learned my addr 172.21.15.88:0/1095225050 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 -- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0320073da0 msgr2=0x7f032015e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0320073da0 0x7f032015e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 -- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f032006ccb0 msgr2=0x7f032006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f032006ccb0 0x7f032006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 -- 172.21.15.88:0/1095225050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0320137240 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f031fe00640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f031800c700 tx=0x7f031800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318018070 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031800ce10 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800a860 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.379+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0320137520 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.380+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0320137ae0 con 0x7f0320147f20 2024-04-26T20:18:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.381+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03200b0350 con 0x7f0320147f20 2024-04-26T20:18:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.382+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f031800aa00 con 0x7f0320147f20 2024-04-26T20:18:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.382+0000 7f0317400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 0x7f02fc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.383+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f031808b380 con 0x7f0320147f20 2024-04-26T20:18:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.383+0000 7f031f400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 0x7f02fc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.383+0000 7f031f400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 0x7f02fc06eed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0310034000 tx=0x7f031002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.387+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03200b0350 con 0x7f0320147f20 2024-04-26T20:18:18.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.585+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0320069650 con 0x7f0320147f20 2024-04-26T20:18:18.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.585+0000 7f0317400640 1 -- 172.21.15.88:0/1095225050 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f03200afed0 con 0x7f0320147f20 2024-04-26T20:18:18.587 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 msgr2=0x7f02fc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 0x7f02fc06eed0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0310034000 tx=0x7f031002c000 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 msgr2=0x7f032015eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f031800c700 tx=0x7f031800cbc0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 shutdown_connections 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0320147f20 0x7f032015eba0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f02fc06ca10 0x7f02fc06eed0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0320073da0 0x7f032015e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 --2- 172.21.15.88:0/1095225050 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f032006ccb0 0x7f032006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 >> 172.21.15.88:0/1095225050 conn(0x7f03200b2d50 msgr2=0x7f0320146fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 shutdown_connections 2024-04-26T20:18:18.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.589+0000 7f0326400640 1 -- 172.21.15.88:0/1095225050 wait complete. 2024-04-26T20:18:18.610 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:18:18.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.808+0000 7f8acbe00640 1 Processor -- start 2024-04-26T20:18:18.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.808+0000 7f8acbe00640 1 -- start start 2024-04-26T20:18:18.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acbe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f8acc15cf40 0x7f8acc0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acbe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc15d970 con 0x7f8acc15c450 2024-04-26T20:18:18.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acbe00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc0b4fe0 con 0x7f8acc15ee60 2024-04-26T20:18:18.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acbe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc0b52c0 con 0x7f8acc15cf40 2024-04-26T20:18:18.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acaa00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f8acc15c450 0x7f8acc15c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47708/0 (socket says 172.21.15.88:47708) 2024-04-26T20:18:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acaa00640 1 -- 172.21.15.88:0/2865054247 learned_addr learned my addr 172.21.15.88:0/2865054247 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8acb400640 1 --2- 172.21.15.88:0/2865054247 >> v2:172.21.15.88:3300/0 conn(0x7f8acc15cf40 0x7f8acc0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8aca000640 1 -- 172.21.15.88:0/2865054247 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928490928 0 0) 0x7f8acc15d970 con 0x7f8acc15c450 2024-04-26T20:18:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.810+0000 7f8aca000640 1 -- 172.21.15.88:0/2865054247 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ab0005730 con 0x7f8acc15c450 2024-04-26T20:18:18.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.811+0000 7f8acb400640 1 -- 172.21.15.88:0/2865054247 >> v1:172.21.15.132:6789/0 conn(0x7f8acc15ee60 legacy=0x7f8acc1639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.811+0000 7f8acb400640 1 -- 172.21.15.88:0/2865054247 >> v1:172.21.15.88:6789/0 conn(0x7f8acc15c450 legacy=0x7f8acc15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.811+0000 7f8acb400640 1 -- 172.21.15.88:0/2865054247 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0b55a0 con 0x7f8acc15cf40 2024-04-26T20:18:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8acb400640 1 --2- 172.21.15.88:0/2865054247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15cf40 0x7f8acc0b4920 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af8dd34e523d6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8aca000640 1 -- 172.21.15.88:0/2865054247 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4031600 con 0x7f8acc15cf40 2024-04-26T20:18:18.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8aca000640 1 -- 172.21.15.88:0/2865054247 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac40317a0 con 0x7f8acc15cf40 2024-04-26T20:18:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8aca000640 1 -- 172.21.15.88:0/2865054247 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4031b70 con 0x7f8acc15cf40 2024-04-26T20:18:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8acbe00640 1 -- 172.21.15.88:0/2865054247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15cf40 msgr2=0x7f8acc0b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.812+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2865054247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15cf40 0x7f8acc0b4920 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009d90 tx=0x7f8ac402a610 comp rx=0 tx=0).stop 2024-04-26T20:18:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.813+0000 7f8acbe00640 1 -- 172.21.15.88:0/2865054247 shutdown_connections 2024-04-26T20:18:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.813+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2865054247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15cf40 0x7f8acc0b4920 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.813+0000 7f8acbe00640 1 -- 172.21.15.88:0/2865054247 >> 172.21.15.88:0/2865054247 conn(0x7f8acc0b2900 msgr2=0x7f8acc0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.813+0000 7f8acbe00640 1 -- 172.21.15.88:0/2865054247 shutdown_connections 2024-04-26T20:18:18.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.813+0000 7f8acbe00640 1 -- 172.21.15.88:0/2865054247 wait complete. 2024-04-26T20:18:18.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.814+0000 7f8acbe00640 1 Processor -- start 2024-04-26T20:18:18.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 -- start start 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8acc15cf40 0x7f8acc155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8acc15ee60 0x7f8acc1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc06e160 con 0x7f8acc15c450 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc06dfe0 con 0x7f8acc15cf40 2024-04-26T20:18:18.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.815+0000 7f8acbe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc06e2e0 con 0x7f8acc15ee60 2024-04-26T20:18:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44156/0 (socket says 172.21.15.88:44156) 2024-04-26T20:18:18.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 -- 172.21.15.88:0/2456436429 learned_addr learned my addr 172.21.15.88:0/2456436429 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8ac2000640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8acc15cf40 0x7f8acc155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 -- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8acc15ee60 msgr2=0x7f8acc1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8acc15ee60 0x7f8acc1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 -- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8acc15cf40 msgr2=0x7f8acc155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8acc15cf40 0x7f8acc155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.816+0000 7f8acaa00640 1 -- 172.21.15.88:0/2456436429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc1488d0 con 0x7f8acc15c450 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8acaa00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c490 tx=0x7f8abc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc017070 con 0x7f8acc15c450 2024-04-26T20:18:18.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc005ce0 con 0x7f8acc15c450 2024-04-26T20:18:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc007290 con 0x7f8acc15c450 2024-04-26T20:18:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc154eb0 con 0x7f8acc15c450 2024-04-26T20:18:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.817+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc0cfe50 con 0x7f8acc15c450 2024-04-26T20:18:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.818+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc0693c0 con 0x7f8acc15c450 2024-04-26T20:18:18.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.819+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f8abc01a070 con 0x7f8acc15c450 2024-04-26T20:18:18.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.820+0000 7f8ac3400640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 0x7f8aa806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:18.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.820+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f8abc08b1b0 con 0x7f8acc15c450 2024-04-26T20:18:18.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.820+0000 7f8ac2000640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 0x7f8aa806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:18.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.821+0000 7f8ac2000640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 0x7f8aa806eed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006fd0 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:18.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:18.825+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8abc059360 con 0x7f8acc15c450 2024-04-26T20:18:19.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.022+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8acc152250 con 0x7f8acc15c450 2024-04-26T20:18:19.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.023+0000 7f8ac3400640 1 -- 172.21.15.88:0/2456436429 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7f8abc058c60 con 0x7f8acc15c450 2024-04-26T20:18:19.024 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 msgr2=0x7f8aa806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 0x7f8aa806eed0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006fd0 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 msgr2=0x7f8acc13e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c490 tx=0x7f8abc00c950 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 shutdown_connections 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f8aa806ca10 0x7f8aa806eed0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f8acc15ee60 0x7f8acc1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8acc15cf40 0x7f8acc155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 --2- 172.21.15.88:0/2456436429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8acc15c450 0x7f8acc13e6a0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 >> 172.21.15.88:0/2456436429 conn(0x7f8acc0b2900 msgr2=0x7f8acc152770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 shutdown_connections 2024-04-26T20:18:19.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.026+0000 7f8acbe00640 1 -- 172.21.15.88:0/2456436429 wait complete. 2024-04-26T20:18:19.047 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:18:19.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.233+0000 7f0cf3a00640 1 Processor -- start 2024-04-26T20:18:19.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.233+0000 7f0cf3a00640 1 -- start start 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf3a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f0cec156650 0x7f0cec15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf3a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f0cec06cbf0 0x7f0cec073830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf3a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec06eea0 con 0x7f0cec06c770 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf3a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec0b4c40 con 0x7f0cec06cbf0 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf3a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec06f140 con 0x7f0cec156650 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0ce3e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f0cec06c770 0x7f0cec073120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47716/0 (socket says 172.21.15.88:47716) 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0ce3e00640 1 -- 172.21.15.88:0/3813096940 learned_addr learned my addr 172.21.15.88:0/3813096940 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:19.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.235+0000 7f0cf1a00640 1 --2- 172.21.15.88:0/3813096940 >> v2:172.21.15.88:3300/0 conn(0x7f0cec06cbf0 0x7f0cec073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cf1000640 1 --2- 172.21.15.88:0/3813096940 >> v2:172.21.15.132:3300/0 conn(0x7f0cec156650 0x7f0cec15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cebe00640 1 -- 172.21.15.88:0/3813096940 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643572636 0 0) 0x7f0cec06eea0 con 0x7f0cec06c770 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cebe00640 1 -- 172.21.15.88:0/3813096940 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd0003520 con 0x7f0cec06c770 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cf1a00640 1 -- 172.21.15.88:0/3813096940 >> v2:172.21.15.132:3300/0 conn(0x7f0cec156650 msgr2=0x7f0cec15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cf1a00640 1 --2- 172.21.15.88:0/3813096940 >> v2:172.21.15.132:3300/0 conn(0x7f0cec156650 0x7f0cec15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cf1a00640 1 -- 172.21.15.88:0/3813096940 >> v1:172.21.15.88:6789/0 conn(0x7f0cec06c770 legacy=0x7f0cec073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.236+0000 7f0cf1a00640 1 -- 172.21.15.88:0/3813096940 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec06f420 con 0x7f0cec06cbf0 2024-04-26T20:18:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.237+0000 7f0cf1a00640 1 --2- 172.21.15.88:0/3813096940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec073830 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0cec075cd0 tx=0x7f0ce402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83efe7b5e713a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.237+0000 7f0cebe00640 1 -- 172.21.15.88:0/3813096940 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4007740 con 0x7f0cec06cbf0 2024-04-26T20:18:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cebe00640 1 -- 172.21.15.88:0/3813096940 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40078e0 con 0x7f0cec06cbf0 2024-04-26T20:18:19.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cebe00640 1 -- 172.21.15.88:0/3813096940 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4007c90 con 0x7f0cec06cbf0 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 -- 172.21.15.88:0/3813096940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 msgr2=0x7f0cec073830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/3813096940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec073830 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0cec075cd0 tx=0x7f0ce402a680 comp rx=0 tx=0).stop 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 -- 172.21.15.88:0/3813096940 shutdown_connections 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/3813096940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec073830 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/3813096940 >> v2:172.21.15.132:3300/0 conn(0x7f0cec156650 0x7f0cec15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 -- 172.21.15.88:0/3813096940 >> 172.21.15.88:0/3813096940 conn(0x7f0cec0b2d50 msgr2=0x7f0cec0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.238+0000 7f0cf3a00640 1 -- 172.21.15.88:0/3813096940 shutdown_connections 2024-04-26T20:18:19.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.239+0000 7f0cf3a00640 1 -- 172.21.15.88:0/3813096940 wait complete. 2024-04-26T20:18:19.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.240+0000 7f0cf3a00640 1 Processor -- start 2024-04-26T20:18:19.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.240+0000 7f0cf3a00640 1 -- start start 2024-04-26T20:18:19.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec06c770 0x7f0cec14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec156650 0x7f0cec15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec06fcc0 con 0x7f0cec06cbf0 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec06f9e0 con 0x7f0cec06c770 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.241+0000 7f0cf3a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec06d040 con 0x7f0cec156650 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44174/0 (socket says 172.21.15.88:44174) 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 -- 172.21.15.88:0/1332208730 learned_addr learned my addr 172.21.15.88:0/1332208730 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0cf1000640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec06c770 0x7f0cec14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 -- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec156650 msgr2=0x7f0cec15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec156650 0x7f0cec15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 -- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec06c770 msgr2=0x7f0cec14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec06c770 0x7f0cec14f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 -- 172.21.15.88:0/1332208730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec157e90 con 0x7f0cec06cbf0 2024-04-26T20:18:19.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce3e00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c430 tx=0x7f0cd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.242+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017070 con 0x7f0cec06cbf0 2024-04-26T20:18:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.243+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd4005ce0 con 0x7f0cec06cbf0 2024-04-26T20:18:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.243+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40072f0 con 0x7f0cec06cbf0 2024-04-26T20:18:19.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.243+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec158170 con 0x7f0cec06cbf0 2024-04-26T20:18:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.243+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec158730 con 0x7f0cec06cbf0 2024-04-26T20:18:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.244+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec069650 con 0x7f0cec06cbf0 2024-04-26T20:18:19.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.245+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f0cd401a070 con 0x7f0cec06cbf0 2024-04-26T20:18:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.245+0000 7f0ce9600640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 0x7f0cc806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.246+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f0cd408b230 con 0x7f0cec06cbf0 2024-04-26T20:18:19.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.246+0000 7f0cf1000640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 0x7f0cc806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.247+0000 7f0cf1000640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 0x7f0cc806eed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.250+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cd4059500 con 0x7f0cec06cbf0 2024-04-26T20:18:19.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.435+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0cec09eeb0 con 0x7f0cec06cbf0 2024-04-26T20:18:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.436+0000 7f0ce9600640 1 -- 172.21.15.88:0/1332208730 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7f0cd4058e00 con 0x7f0cec06cbf0 2024-04-26T20:18:19.438 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:18:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 msgr2=0x7f0cc806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 0x7f0cc806eed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc008040 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 msgr2=0x7f0cec157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c430 tx=0x7f0cd400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 shutdown_connections 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f0cc806ca10 0x7f0cc806eed0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f0cec156650 0x7f0cec15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cec06cbf0 0x7f0cec157440 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 --2- 172.21.15.88:0/1332208730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0cec06c770 0x7f0cec14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 >> 172.21.15.88:0/1332208730 conn(0x7f0cec0b2d50 msgr2=0x7f0cec072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 shutdown_connections 2024-04-26T20:18:19.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.439+0000 7f0cf3a00640 1 -- 172.21.15.88:0/1332208730 wait complete. 2024-04-26T20:18:19.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:18:19.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.642+0000 7f4ab6600640 1 Processor -- start 2024-04-26T20:18:19.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.642+0000 7f4ab6600640 1 -- start start 2024-04-26T20:18:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.644+0000 7f4ab6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 0x7f4ab006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.644+0000 7f4ab6600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4ab00b5d40 0x7f4ab006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.644+0000 7f4ab6600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab00727f0 con 0x7f4ab00d0e70 2024-04-26T20:18:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.644+0000 7f4ab6600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4ab00b4a10 con 0x7f4ab00b5d40 2024-04-26T20:18:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.644+0000 7f4ab6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab00b4cf0 con 0x7f4ab00b5660 2024-04-26T20:18:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.645+0000 7f4aa7e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 0x7f4ab006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.645+0000 7f4ab5200640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f4ab00d0e70 0x7f4ab00d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:52932/0 (socket says 172.21.15.88:52932) 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.645+0000 7f4ab5200640 1 -- 172.21.15.88:0/1942638026 learned_addr learned my addr 172.21.15.88:0/1942638026 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.645+0000 7f4ab5c00640 1 --2- 172.21.15.88:0/1942638026 >> v2:172.21.15.132:3300/0 conn(0x7f4ab00b5d40 0x7f4ab006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.646+0000 7f4aafe00640 1 -- 172.21.15.88:0/1942638026 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1038437268 0 0) 0x7f4ab00727f0 con 0x7f4ab00d0e70 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.646+0000 7f4aa7e00640 1 -- 172.21.15.88:0/1942638026 >> v2:172.21.15.132:3300/0 conn(0x7f4ab00b5d40 msgr2=0x7f4ab006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.646+0000 7f4aa7e00640 1 --2- 172.21.15.88:0/1942638026 >> v2:172.21.15.132:3300/0 conn(0x7f4ab00b5d40 0x7f4ab006e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.646+0000 7f4aa7e00640 1 -- 172.21.15.88:0/1942638026 >> v1:172.21.15.132:6789/0 conn(0x7f4ab00d0e70 legacy=0x7f4ab00d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.646+0000 7f4aa7e00640 1 -- 172.21.15.88:0/1942638026 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab00b4fd0 con 0x7f4ab00b5660 2024-04-26T20:18:19.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.647+0000 7f4aa7e00640 1 --2- 172.21.15.88:0/1942638026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 0x7f4ab006e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097a0 tx=0x7f4aa002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2936ab9ce3d7041 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.648+0000 7f4aafe00640 1 -- 172.21.15.88:0/1942638026 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0037070 con 0x7f4ab00b5660 2024-04-26T20:18:19.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.648+0000 7f4aafe00640 1 -- 172.21.15.88:0/1942638026 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa002ae50 con 0x7f4ab00b5660 2024-04-26T20:18:19.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.648+0000 7f4aafe00640 1 -- 172.21.15.88:0/1942638026 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0032660 con 0x7f4ab00b5660 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 -- 172.21.15.88:0/1942638026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 msgr2=0x7f4ab006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 --2- 172.21.15.88:0/1942638026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 0x7f4ab006e3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097a0 tx=0x7f4aa002a100 comp rx=0 tx=0).stop 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 -- 172.21.15.88:0/1942638026 shutdown_connections 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 --2- 172.21.15.88:0/1942638026 >> v2:172.21.15.132:3300/0 conn(0x7f4ab00b5d40 0x7f4ab006e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 --2- 172.21.15.88:0/1942638026 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00b5660 0x7f4ab006e3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 -- 172.21.15.88:0/1942638026 >> 172.21.15.88:0/1942638026 conn(0x7f4ab006c740 msgr2=0x7f4ab006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 -- 172.21.15.88:0/1942638026 shutdown_connections 2024-04-26T20:18:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.649+0000 7f4ab6600640 1 -- 172.21.15.88:0/1942638026 wait complete. 2024-04-26T20:18:19.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.651+0000 7f4ab6600640 1 Processor -- start 2024-04-26T20:18:19.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.651+0000 7f4ab6600640 1 -- start start 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.652+0000 7f4ab6600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ab00b5660 0x7f4ab01f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.652+0000 7f4ab6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4ab6600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00d0e70 0x7f4ab015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4ab6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab015f320 con 0x7f4ab00b5d40 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4ab6600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab015f1a0 con 0x7f4ab00b5660 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4ab6600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab006ee70 con 0x7f4ab00d0e70 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44186/0 (socket says 172.21.15.88:44186) 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 -- 172.21.15.88:0/2218012796 learned_addr learned my addr 172.21.15.88:0/2218012796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:19.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4ab5200640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ab00b5660 0x7f4ab01f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 -- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00d0e70 msgr2=0x7f4ab015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00d0e70 0x7f4ab015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 -- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ab00b5660 msgr2=0x7f4ab01f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ab00b5660 0x7f4ab01f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.653+0000 7f4aa7e00640 1 -- 172.21.15.88:0/2218012796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab0149920 con 0x7f4ab00b5d40 2024-04-26T20:18:19.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4aa7e00640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0035040 tx=0x7f4aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0041070 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aa003c460 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0037040 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab015f640 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.654+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01f5080 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.655+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab00693c0 con 0x7f4ab00b5d40 2024-04-26T20:18:19.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.656+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f4aa003a070 con 0x7f4ab00b5d40 2024-04-26T20:18:19.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.657+0000 7f4aad600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 0x7f4a8c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:19.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.657+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f4aa00abf90 con 0x7f4ab00b5d40 2024-04-26T20:18:19.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.657+0000 7f4ab5200640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 0x7f4a8c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:19.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.658+0000 7f4ab5200640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 0x7f4a8c06eed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a98000f60 tx=0x7f4a98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:19.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.663+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4aa007a1d0 con 0x7f4ab00b5d40 2024-04-26T20:18:19.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.880+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4ab00afd40 con 0x7f4ab00b5d40 2024-04-26T20:18:19.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.881+0000 7f4aad600640 1 -- 172.21.15.88:0/2218012796 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7f4aa0079ad0 con 0x7f4ab00b5d40 2024-04-26T20:18:19.883 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 msgr2=0x7f4a8c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 0x7f4a8c06eed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4a98000f60 tx=0x7f4a98009040 comp rx=0 tx=0).stop 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 msgr2=0x7f4ab015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0035040 tx=0x7f4aa00046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 shutdown_connections 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4a8c06ca10 0x7f4a8c06eed0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ab00d0e70 0x7f4ab015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ab00b5d40 0x7f4ab015b270 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 --2- 172.21.15.88:0/2218012796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ab00b5660 0x7f4ab01f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:19.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 >> 172.21.15.88:0/2218012796 conn(0x7f4ab006c740 msgr2=0x7f4ab015dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:19.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 shutdown_connections 2024-04-26T20:18:19.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:19.884+0000 7f4ab6600640 1 -- 172.21.15.88:0/2218012796 wait complete. 2024-04-26T20:18:19.906 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi132.front.sepia.ceph.com... 2024-04-26T20:18:19.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:19.906 DEBUG:teuthology.orchestra.run.smithi132:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:18:19.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[41337]: fuse finished with error 0 and tester_r 0 2024-04-26T20:18:19.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:18:19.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:19.996 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:18:20.107 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:18:20.107 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:18:20.107 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:20.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:18:20.165 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:18:20.165 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:18:20.165 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:20.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:18:20.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:18:20.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 Processor -- start 2024-04-26T20:18:20.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 -- start start 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006ce80 0x7f53f01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f53f015f740 0x7f53f015fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f00b5fb0 con 0x7f53f01598a0 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f53f0159c80 con 0x7f53f015f740 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.426+0000 7f53f7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f53f0160110 con 0x7f53f006ce80 2024-04-26T20:18:20.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f53f01598a0 0x7f53f006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47726/0 (socket says 172.21.15.88:47726) 2024-04-26T20:18:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5200640 1 -- 172.21.15.88:0/3097295646 learned_addr learned my addr 172.21.15.88:0/3097295646 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5c00640 1 --2- 172.21.15.88:0/3097295646 >> v2:172.21.15.88:3300/0 conn(0x7f53f015f740 0x7f53f015fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53e7e00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006ce80 0x7f53f01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53efe00640 1 -- 172.21.15.88:0/3097295646 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 41291939 0 0) 0x7f53f00b5fb0 con 0x7f53f01598a0 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53efe00640 1 -- 172.21.15.88:0/3097295646 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d4003520 con 0x7f53f01598a0 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5c00640 1 -- 172.21.15.88:0/3097295646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006ce80 msgr2=0x7f53f01638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5c00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006ce80 0x7f53f01638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5c00640 1 -- 172.21.15.88:0/3097295646 >> v1:172.21.15.88:6789/0 conn(0x7f53f01598a0 legacy=0x7f53f006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.427+0000 7f53f5c00640 1 -- 172.21.15.88:0/3097295646 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0160370 con 0x7f53f015f740 2024-04-26T20:18:20.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.428+0000 7f53f5c00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f015f740 0x7f53f015fbd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009dc0 tx=0x7f53e802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef0dbea122bb466 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53efe00640 1 -- 172.21.15.88:0/3097295646 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007740 con 0x7f53f015f740 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53efe00640 1 -- 172.21.15.88:0/3097295646 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e80078e0 con 0x7f53f015f740 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53efe00640 1 -- 172.21.15.88:0/3097295646 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007c90 con 0x7f53f015f740 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 -- 172.21.15.88:0/3097295646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f015f740 msgr2=0x7f53f015fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f015f740 0x7f53f015fbd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009dc0 tx=0x7f53e802a5f0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 -- 172.21.15.88:0/3097295646 shutdown_connections 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f015f740 0x7f53f015fbd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3097295646 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f006ce80 0x7f53f01638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 -- 172.21.15.88:0/3097295646 >> 172.21.15.88:0/3097295646 conn(0x7f53f00b2d50 msgr2=0x7f53f00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 -- 172.21.15.88:0/3097295646 shutdown_connections 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.429+0000 7f53f7c00640 1 -- 172.21.15.88:0/3097295646 wait complete. 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.431+0000 7f53f7c00640 1 Processor -- start 2024-04-26T20:18:20.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.431+0000 7f53f7c00640 1 -- start start 2024-04-26T20:18:20.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 0x7f53f00c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f015f740 0x7f53f006f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53f0163390 con 0x7f53f006ce80 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01630b0 con 0x7f53f015f740 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f7c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f53f0160650 con 0x7f53f01598a0 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.432+0000 7f53f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44208/0 (socket says 172.21.15.88:44208) 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 -- 172.21.15.88:0/3235575591 learned_addr learned my addr 172.21.15.88:0/3235575591 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53e7e00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 0x7f53f00c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f015f740 0x7f53f006f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 -- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 msgr2=0x7f53f00c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 0x7f53f00c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 -- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f015f740 msgr2=0x7f53f006f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f015f740 0x7f53f006f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 -- 172.21.15.88:0/3235575591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f01f5000 con 0x7f53f006ce80 2024-04-26T20:18:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53e7e00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 0x7f53f00c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:20.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.433+0000 7f53f5200640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c430 tx=0x7f53e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.434+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0017070 con 0x7f53f006ce80 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.434+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0005ce0 con 0x7f53f006ce80 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.434+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f01f52e0 con 0x7f53f006ce80 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.434+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00072f0 con 0x7f53f006ce80 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.434+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01f58a0 con 0x7f53f006ce80 2024-04-26T20:18:20.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.435+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f0069540 con 0x7f53f006ce80 2024-04-26T20:18:20.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.436+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f53e001a070 con 0x7f53f006ce80 2024-04-26T20:18:20.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.437+0000 7f53ed600640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.437+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f53e008b230 con 0x7f53f006ce80 2024-04-26T20:18:20.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.437+0000 7f53e7e00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.438+0000 7f53e7e00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 0x7f53cc06eed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53f015f530 tx=0x7f53d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.441+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53e0059500 con 0x7f53f006ce80 2024-04-26T20:18:20.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.683+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53f0073f90 con 0x7f53f006ce80 2024-04-26T20:18:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.684+0000 7f53ed600640 1 -- 172.21.15.88:0/3235575591 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5186 (secure 0 0 0) 0x7f53e0058e00 con 0x7f53f006ce80 2024-04-26T20:18:20.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:20.686 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:18:20.686 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 msgr2=0x7f53cc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 0x7f53cc06eed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53f015f530 tx=0x7f53d8008040 comp rx=0 tx=0).stop 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 msgr2=0x7f53f0148640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c430 tx=0x7f53e000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 shutdown_connections 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f015f740 0x7f53f006f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f53cc06ca10 0x7f53cc06eed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f53f01598a0 0x7f53f00c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 --2- 172.21.15.88:0/3235575591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53f006ce80 0x7f53f0148640 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 >> 172.21.15.88:0/3235575591 conn(0x7f53f00b2d50 msgr2=0x7f53f0072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:20.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 shutdown_connections 2024-04-26T20:18:20.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.687+0000 7f53f7c00640 1 -- 172.21.15.88:0/3235575591 wait complete. 2024-04-26T20:18:20.709 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.898+0000 7f56afa00640 1 Processor -- start 2024-04-26T20:18:20.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.898+0000 7f56afa00640 1 -- start start 2024-04-26T20:18:20.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56afa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 0x7f56a80add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56afa00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f56a80cd650 0x7f56a806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56afa00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a80af9a0 con 0x7f56a80b4f50 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56afa00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f56a80ae2c0 con 0x7f56a80cd650 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56afa00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f56a80b05b0 con 0x7f56a815cdf0 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56ada00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f56a80b4f50 0x7f56a8072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47738/0 (socket says 172.21.15.88:47738) 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56ada00640 1 -- 172.21.15.88:0/2895439000 learned_addr learned my addr 172.21.15.88:0/2895439000 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f569fe00640 1 --2- 172.21.15.88:0/2895439000 >> v2:172.21.15.88:3300/0 conn(0x7f56a80cd650 0x7f56a806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.899+0000 7f56ad000640 1 --2- 172.21.15.88:0/2895439000 >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 0x7f56a80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f56a7e00640 1 -- 172.21.15.88:0/2895439000 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1454412793 0 0) 0x7f56a80af9a0 con 0x7f56a80b4f50 2024-04-26T20:18:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f56a7e00640 1 -- 172.21.15.88:0/2895439000 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f568c003520 con 0x7f56a80b4f50 2024-04-26T20:18:20.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f569fe00640 1 -- 172.21.15.88:0/2895439000 >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 msgr2=0x7f56a80add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f569fe00640 1 --2- 172.21.15.88:0/2895439000 >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 0x7f56a80add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f569fe00640 1 -- 172.21.15.88:0/2895439000 >> v1:172.21.15.88:6789/0 conn(0x7f56a80b4f50 legacy=0x7f56a8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f569fe00640 1 -- 172.21.15.88:0/2895439000 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a80afc80 con 0x7f56a80cd650 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.900+0000 7f56ad000640 1 --2- 172.21.15.88:0/2895439000 >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 0x7f56a80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.901+0000 7f569fe00640 1 --2- 172.21.15.88:0/2895439000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a80cd650 0x7f56a806c770 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f56a80b3950 tx=0x7f569002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8451a14e0525ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.901+0000 7f56a7e00640 1 -- 172.21.15.88:0/2895439000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037070 con 0x7f56a80cd650 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.901+0000 7f56a7e00640 1 -- 172.21.15.88:0/2895439000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690032440 con 0x7f56a80cd650 2024-04-26T20:18:20.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.901+0000 7f56a7e00640 1 -- 172.21.15.88:0/2895439000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690032810 con 0x7f56a80cd650 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 -- 172.21.15.88:0/2895439000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a80cd650 msgr2=0x7f56a806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 --2- 172.21.15.88:0/2895439000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a80cd650 0x7f56a806c770 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f56a80b3950 tx=0x7f569002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 -- 172.21.15.88:0/2895439000 shutdown_connections 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 --2- 172.21.15.88:0/2895439000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a80cd650 0x7f56a806c770 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 --2- 172.21.15.88:0/2895439000 >> v2:172.21.15.132:3300/0 conn(0x7f56a815cdf0 0x7f56a80add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 -- 172.21.15.88:0/2895439000 >> 172.21.15.88:0/2895439000 conn(0x7f56a80b2d50 msgr2=0x7f56a80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:20.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 -- 172.21.15.88:0/2895439000 shutdown_connections 2024-04-26T20:18:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.902+0000 7f56afa00640 1 -- 172.21.15.88:0/2895439000 wait complete. 2024-04-26T20:18:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.904+0000 7f56afa00640 1 Processor -- start 2024-04-26T20:18:20.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.904+0000 7f56afa00640 1 -- start start 2024-04-26T20:18:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56a80b4f50 0x7f56a81517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a80cd650 0x7f56a809c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56a815d1d0 con 0x7f56a815cdf0 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f56a80b0240 con 0x7f56a80cd650 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56afa00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f56a815d350 con 0x7f56a80b4f50 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44224/0 (socket says 172.21.15.88:44224) 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 -- 172.21.15.88:0/433421391 learned_addr learned my addr 172.21.15.88:0/433421391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f569fe00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a80cd650 0x7f56a809c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 -- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56a80b4f50 msgr2=0x7f56a81517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56a80b4f50 0x7f56a81517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 -- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a80cd650 msgr2=0x7f56a809c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a80cd650 0x7f56a809c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.905+0000 7f56ada00640 1 -- 172.21.15.88:0/433421391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a81f4df0 con 0x7f56a815cdf0 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56ada00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f56a000cb70 tx=0x7f56a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0017070 con 0x7f56a815cdf0 2024-04-26T20:18:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a000ad40 con 0x7f56a815cdf0 2024-04-26T20:18:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0012620 con 0x7f56a815cdf0 2024-04-26T20:18:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a81f50d0 con 0x7f56a815cdf0 2024-04-26T20:18:20.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.906+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81f5690 con 0x7f56a815cdf0 2024-04-26T20:18:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.907+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a8069540 con 0x7f56a815cdf0 2024-04-26T20:18:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.908+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f56a001a070 con 0x7f56a815cdf0 2024-04-26T20:18:20.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.909+0000 7f56a5600640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 0x7f568406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.909+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f56a008bc30 con 0x7f56a815cdf0 2024-04-26T20:18:20.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.910+0000 7f56ad000640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 0x7f568406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:20.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.910+0000 7f56ad000640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 0x7f568406eed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5698006e80 tx=0x7f5698008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:20.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:18:20.916 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:18:20.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:20.914+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56a0059f00 con 0x7f56a815cdf0 2024-04-26T20:18:21.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.117+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a80b4960 con 0x7f56a815cdf0 2024-04-26T20:18:21.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.119+0000 7f56a5600640 1 -- 172.21.15.88:0/433421391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16754 (secure 0 0 0) 0x7f56a00600d0 con 0x7f56a815cdf0 2024-04-26T20:18:21.121 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:21.121 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:03:29.112906+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:21.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.121+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 msgr2=0x7f568406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.121+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 0x7f568406eed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5698006e80 tx=0x7f5698008040 comp rx=0 tx=0).stop 2024-04-26T20:18:21.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.121+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 msgr2=0x7f56a815dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.121+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f56a000cb70 tx=0x7f56a000a430 comp rx=0 tx=0).stop 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 shutdown_connections 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56a815cdf0 0x7f56a815dc40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56a80cd650 0x7f56a809c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f568406ca10 0x7f568406eed0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 --2- 172.21.15.88:0/433421391 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f56a80b4f50 0x7f56a81517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 >> 172.21.15.88:0/433421391 conn(0x7f56a80b2d50 msgr2=0x7f56a80d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 shutdown_connections 2024-04-26T20:18:21.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.122+0000 7f56afa00640 1 -- 172.21.15.88:0/433421391 wait complete. 2024-04-26T20:18:21.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:59:34.974341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:02:41.445900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:02:42.613083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:18:21.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:18:21.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.346+0000 7f1bdc800640 1 Processor -- start 2024-04-26T20:18:21.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.346+0000 7f1bdc800640 1 -- start start 2024-04-26T20:18:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bdc800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1bd406ce80 0x7f1bd40b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bdc800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd4073da0 0x7f1bd40aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bdc800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd40ade20 con 0x7f1bd4147f20 2024-04-26T20:18:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bdc800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd40ae100 con 0x7f1bd406ce80 2024-04-26T20:18:21.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bdc800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd40ae3e0 con 0x7f1bd4073da0 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bd9400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1bd406ce80 0x7f1bd40b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bd9400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1bd406ce80 0x7f1bd40b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44230/0 (socket says 172.21.15.88:44230) 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.348+0000 7f1bd9400640 1 -- 172.21.15.88:0/1820112132 learned_addr learned my addr 172.21.15.88:0/1820112132 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.349+0000 7f1bd9400640 1 -- 172.21.15.88:0/1820112132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd4073da0 msgr2=0x7f1bd40aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.349+0000 7f1bd9400640 1 --2- 172.21.15.88:0/1820112132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd4073da0 0x7f1bd40aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.349+0000 7f1bd9400640 1 -- 172.21.15.88:0/1820112132 >> v1:172.21.15.132:6789/0 conn(0x7f1bd4147f20 legacy=0x7f1bd406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.349+0000 7f1bd9400640 1 -- 172.21.15.88:0/1820112132 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd40ae6c0 con 0x7f1bd406ce80 2024-04-26T20:18:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.350+0000 7f1bd9400640 1 --2- 172.21.15.88:0/1820112132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd406ce80 0x7f1bd40b5c20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3489139d57499b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.350+0000 7f1bd8a00640 1 -- 172.21.15.88:0/1820112132 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bd406ce80 2024-04-26T20:18:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.350+0000 7f1bd8a00640 1 -- 172.21.15.88:0/1820112132 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8032440 con 0x7f1bd406ce80 2024-04-26T20:18:21.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.350+0000 7f1bd8a00640 1 -- 172.21.15.88:0/1820112132 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8032810 con 0x7f1bd406ce80 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 -- 172.21.15.88:0/1820112132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd406ce80 msgr2=0x7f1bd40b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 --2- 172.21.15.88:0/1820112132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd406ce80 0x7f1bd40b5c20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc802a210 comp rx=0 tx=0).stop 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 -- 172.21.15.88:0/1820112132 shutdown_connections 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 --2- 172.21.15.88:0/1820112132 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd4073da0 0x7f1bd40aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 --2- 172.21.15.88:0/1820112132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd406ce80 0x7f1bd40b5c20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 -- 172.21.15.88:0/1820112132 >> 172.21.15.88:0/1820112132 conn(0x7f1bd40b2d50 msgr2=0x7f1bd40b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 -- 172.21.15.88:0/1820112132 shutdown_connections 2024-04-26T20:18:21.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.351+0000 7f1bdc800640 1 -- 172.21.15.88:0/1820112132 wait complete. 2024-04-26T20:18:21.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.353+0000 7f1bdc800640 1 Processor -- start 2024-04-26T20:18:21.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.353+0000 7f1bdc800640 1 -- start start 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 0x7f1bd41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 0x7f1bd415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd406e220 con 0x7f1bd4073da0 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd40741d0 con 0x7f1bd4147f20 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bdc800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd40af400 con 0x7f1bd406ce80 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44244/0 (socket says 172.21.15.88:44244) 2024-04-26T20:18:21.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.354+0000 7f1bd9400640 1 -- 172.21.15.88:0/3477399451 learned_addr learned my addr 172.21.15.88:0/3477399451 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:21.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bda800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 0x7f1bd415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9e00640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 0x7f1bd41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 -- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 msgr2=0x7f1bd41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 0x7f1bd41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 -- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 msgr2=0x7f1bd415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 0x7f1bd415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 -- 172.21.15.88:0/3477399451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd40cb000 con 0x7f1bd4073da0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bda800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 0x7f1bd415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9e00640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 0x7f1bd41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bd9400640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8041070 con 0x7f1bd4073da0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc803c870 con 0x7f1bd4073da0 2024-04-26T20:18:21.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.355+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037040 con 0x7f1bd4073da0 2024-04-26T20:18:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.356+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd40cfaf0 con 0x7f1bd4073da0 2024-04-26T20:18:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.356+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd40cb2c0 con 0x7f1bd4073da0 2024-04-26T20:18:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.357+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd4069650 con 0x7f1bd4073da0 2024-04-26T20:18:21.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.358+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f1bc8044070 con 0x7f1bd4073da0 2024-04-26T20:18:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.359+0000 7f1bce000640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 0x7f1bb806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.359+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f1bc80ac9d0 con 0x7f1bd4073da0 2024-04-26T20:18:21.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.359+0000 7f1bd9e00640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 0x7f1bb806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.359+0000 7f1bd9e00640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 0x7f1bb806eed0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400a830 tx=0x7f1bc4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.363+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bc807ad30 con 0x7f1bd4073da0 2024-04-26T20:18:21.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.565+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bd40b24e0 con 0x7f1bd4073da0 2024-04-26T20:18:21.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.566+0000 7f1bce000640 1 -- 172.21.15.88:0/3477399451 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f1bc807a630 con 0x7f1bd4073da0 2024-04-26T20:18:21.566 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-04-26T20:18:21.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.568+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 msgr2=0x7f1bb806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.568+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 0x7f1bb806eed0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400a830 tx=0x7f1bc4008040 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.568+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 msgr2=0x7f1bd415e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.568+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009870 tx=0x7f1bc80046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 shutdown_connections 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1bd4147f20 0x7f1bd415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bd4073da0 0x7f1bd415e5f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1bb806ca10 0x7f1bb806eed0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 --2- 172.21.15.88:0/3477399451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1bd406ce80 0x7f1bd41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 >> 172.21.15.88:0/3477399451 conn(0x7f1bd40b2d50 msgr2=0x7f1bd40b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 shutdown_connections 2024-04-26T20:18:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.569+0000 7f1bdc800640 1 -- 172.21.15.88:0/3477399451 wait complete. 2024-04-26T20:18:21.588 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:18:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.775+0000 7fa947200640 1 Processor -- start 2024-04-26T20:18:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.775+0000 7fa947200640 1 -- start start 2024-04-26T20:18:21.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.777+0000 7fa947200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa94006ce80 0x7fa9400b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.777+0000 7fa947200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400af800 con 0x7fa9400b5ca0 2024-04-26T20:18:21.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.777+0000 7fa947200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400ae340 con 0x7fa94015d100 2024-04-26T20:18:21.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.777+0000 7fa947200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa9400afaa0 con 0x7fa94006ce80 2024-04-26T20:18:21.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.778+0000 7fa945200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa9400b5ca0 0x7fa9400b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47746/0 (socket says 172.21.15.88:47746) 2024-04-26T20:18:21.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.778+0000 7fa945200640 1 -- 172.21.15.88:0/2837541455 learned_addr learned my addr 172.21.15.88:0/2837541455 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:21.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.778+0000 7fa937e00640 1 --2- 172.21.15.88:0/2837541455 >> v2:172.21.15.132:3300/0 conn(0x7fa94006ce80 0x7fa9400b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.778+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652540528 0 0) 0x7fa9400af800 con 0x7fa9400b5ca0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.778+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa924003520 con 0x7fa9400b5ca0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4061599091 0 0) 0x7fa924003520 con 0x7fa9400b5ca0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 >> v2:172.21.15.132:3300/0 conn(0x7fa94006ce80 msgr2=0x7fa9400b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 --2- 172.21.15.88:0/2837541455 >> v2:172.21.15.132:3300/0 conn(0x7fa94006ce80 0x7fa9400b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 >> v1:172.21.15.132:6789/0 conn(0x7fa94015d100 legacy=0x7fa94006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400afd80 con 0x7fa9400b5ca0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa9380019f0 con 0x7fa9400b5ca0 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 >> v1:172.21.15.88:6789/0 conn(0x7fa9400b5ca0 legacy=0x7fa9400b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa924003a00 0x7fa924003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa924004800 0x7fa924004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa92400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400afaa0 con 0x7fa924005210 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa92400c160 con 0x7fa924003a00 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa92400c360 con 0x7fa924004800 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa945200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa92400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.779+0000 7fa937e00640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa924004800 0x7fa924004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 -- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa924004800 msgr2=0x7fa924004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa924004800 0x7fa924004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 -- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa924003a00 msgr2=0x7fa924003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:21.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa924003a00 0x7fa924003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 -- 172.21.15.88:0/2837541455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa924003520 con 0x7fa924005210 2024-04-26T20:18:21.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.780+0000 7fa945200640 1 -- 172.21.15.88:0/2837541455 reap_dead start 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa945200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa92400b9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa93800a760 tx=0x7fa924005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45b05fa3661b01a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93800a3b0 con 0x7fa924005210 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93802f070 con 0x7fa924005210 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa93f400640 1 -- 172.21.15.88:0/2837541455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93802bba0 con 0x7fa924005210 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa947200640 1 -- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 msgr2=0x7fa92400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.781+0000 7fa947200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa92400b9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa93800a760 tx=0x7fa924005830 comp rx=0 tx=0).stop 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.782+0000 7fa947200640 1 -- 172.21.15.88:0/2837541455 shutdown_connections 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.782+0000 7fa947200640 1 --2- 172.21.15.88:0/2837541455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa92400b9c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.782+0000 7fa947200640 1 -- 172.21.15.88:0/2837541455 >> 172.21.15.88:0/2837541455 conn(0x7fa9400b2d50 msgr2=0x7fa9400b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:21.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.782+0000 7fa947200640 1 -- 172.21.15.88:0/2837541455 shutdown_connections 2024-04-26T20:18:21.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.782+0000 7fa947200640 1 -- 172.21.15.88:0/2837541455 wait complete. 2024-04-26T20:18:21.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.783+0000 7fa947200640 1 Processor -- start 2024-04-26T20:18:21.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.784+0000 7fa947200640 1 -- start start 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.784+0000 7fa947200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa947200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa94009dc00 0x7fa9401f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa947200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9401f8740 0x7fa940150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa947200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa94009ee50 con 0x7fa924005210 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa947200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400b0340 con 0x7fa9401f8740 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa947200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa940160b70 con 0x7fa94009dc00 2024-04-26T20:18:21.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44266/0 (socket says 172.21.15.88:44266) 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 -- 172.21.15.88:0/1746777044 learned_addr learned my addr 172.21.15.88:0/1746777044 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa937e00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa94009dc00 0x7fa9401f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa945200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9401f8740 0x7fa940150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 -- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa94009dc00 msgr2=0x7fa9401f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa94009dc00 0x7fa9401f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 -- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9401f8740 msgr2=0x7fa940150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9401f8740 0x7fa940150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.785+0000 7fa93fe00640 1 -- 172.21.15.88:0/1746777044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94009e180 con 0x7fa924005210 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa93fe00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c430 tx=0x7fa93000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930017070 con 0x7fa924005210 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9300052f0 con 0x7fa924005210 2024-04-26T20:18:21.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94009e460 con 0x7fa924005210 2024-04-26T20:18:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930003830 con 0x7fa924005210 2024-04-26T20:18:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.786+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9401f6580 con 0x7fa924005210 2024-04-26T20:18:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.787+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa940069650 con 0x7fa924005210 2024-04-26T20:18:21.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.789+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fa93001a070 con 0x7fa924005210 2024-04-26T20:18:21.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.789+0000 7fa93cc00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 0x7fa91c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:21.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.790+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fa93008b140 con 0x7fa924005210 2024-04-26T20:18:21.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.790+0000 7fa937e00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 0x7fa91c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:21.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.790+0000 7fa937e00640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 0x7fa91c06eed0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa94009e870 tx=0x7fa928008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:21.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.794+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa930059410 con 0x7fa924005210 2024-04-26T20:18:21.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.996+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9401f8b20 con 0x7fa924005210 2024-04-26T20:18:22.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:21.997+0000 7fa93cc00640 1 -- 172.21.15.88:0/1746777044 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7fa930058d10 con 0x7fa924005210 2024-04-26T20:18:22.002 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-04-26T20:18:22.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 msgr2=0x7fa91c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 0x7fa91c06eed0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa94009e870 tx=0x7fa928008040 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 msgr2=0x7fa94009d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c430 tx=0x7fa93000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 shutdown_connections 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9401f8740 0x7fa940150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa91c06ca10 0x7fa91c06eed0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa94009dc00 0x7fa9401f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 --2- 172.21.15.88:0/1746777044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa924005210 0x7fa94009d6c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.000+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 >> 172.21.15.88:0/1746777044 conn(0x7fa9400b2d50 msgr2=0x7fa9400d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.001+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 shutdown_connections 2024-04-26T20:18:22.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.001+0000 7fa947200640 1 -- 172.21.15.88:0/1746777044 wait complete. 2024-04-26T20:18:22.020 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:18:22.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.223+0000 7f324be00640 1 Processor -- start 2024-04-26T20:18:22.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.224+0000 7f324be00640 1 -- start start 2024-04-26T20:18:22.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.225+0000 7f324be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f324c14ce70 0x7f324c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.225+0000 7f324be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c0d1470 0x7f324c166f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.225+0000 7f324be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324c1675f0 con 0x7f324c163920 2024-04-26T20:18:22.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.225+0000 7f324be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f324c06fad0 con 0x7f324c14ce70 2024-04-26T20:18:22.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.225+0000 7f324be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f324c167e90 con 0x7f324c0d1470 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.226+0000 7f324aa00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f324c14ce70 0x7f324c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.226+0000 7f3243e00640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7f324c163920 0x7f324c0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:52956/0 (socket says 172.21.15.88:52956) 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.226+0000 7f3243e00640 1 -- 172.21.15.88:0/1397417452 learned_addr learned my addr 172.21.15.88:0/1397417452 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.226+0000 7f324b400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c0d1470 0x7f324c166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.227+0000 7f324aa00640 1 -- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c0d1470 msgr2=0x7f324c166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.227+0000 7f324aa00640 1 --2- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c0d1470 0x7f324c166f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.227+0000 7f324aa00640 1 -- 172.21.15.88:0/1397417452 >> v1:172.21.15.132:6789/0 conn(0x7f324c163920 legacy=0x7f324c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.227+0000 7f324aa00640 1 -- 172.21.15.88:0/1397417452 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c1678d0 con 0x7f324c14ce70 2024-04-26T20:18:22.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.228+0000 7f324aa00640 1 --2- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c14ce70 0x7f324c0b5ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f323c0097a0 tx=0x7f323c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874ee59c7c4fcb13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.229+0000 7f324a000640 1 -- 172.21.15.88:0/1397417452 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c037070 con 0x7f324c14ce70 2024-04-26T20:18:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.229+0000 7f324a000640 1 -- 172.21.15.88:0/1397417452 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c032440 con 0x7f324c14ce70 2024-04-26T20:18:22.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.229+0000 7f324a000640 1 -- 172.21.15.88:0/1397417452 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c032810 con 0x7f324c14ce70 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 -- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c14ce70 msgr2=0x7f324c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 --2- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c14ce70 0x7f324c0b5ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f323c0097a0 tx=0x7f323c02a180 comp rx=0 tx=0).stop 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 -- 172.21.15.88:0/1397417452 shutdown_connections 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 --2- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c0d1470 0x7f324c166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 --2- 172.21.15.88:0/1397417452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c14ce70 0x7f324c0b5ad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 -- 172.21.15.88:0/1397417452 >> 172.21.15.88:0/1397417452 conn(0x7f324c06c530 msgr2=0x7f324c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 -- 172.21.15.88:0/1397417452 shutdown_connections 2024-04-26T20:18:22.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.230+0000 7f324be00640 1 -- 172.21.15.88:0/1397417452 wait complete. 2024-04-26T20:18:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.232+0000 7f324be00640 1 Processor -- start 2024-04-26T20:18:22.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.232+0000 7f324be00640 1 -- start start 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c14ce70 0x7f324c140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c141300 0x7f324c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f324c14d250 con 0x7f324c0d1470 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f324c072300 con 0x7f324c141300 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.233+0000 7f324be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f324c072790 con 0x7f324c14ce70 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44282/0 (socket says 172.21.15.88:44282) 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 -- 172.21.15.88:0/3276610034 learned_addr learned my addr 172.21.15.88:0/3276610034 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f3243e00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c14ce70 0x7f324c140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324b400640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c141300 0x7f324c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 -- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c14ce70 msgr2=0x7f324c140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c14ce70 0x7f324c140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 -- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c141300 msgr2=0x7f324c13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c141300 0x7f324c13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.234+0000 7f324aa00640 1 -- 172.21.15.88:0/3276610034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c140920 con 0x7f324c0d1470 2024-04-26T20:18:22.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f324aa00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f323c02a770 tx=0x7f323c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c037050 con 0x7f324c0d1470 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c043070 con 0x7f324c0d1470 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c03cb30 con 0x7f324c0d1470 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c142c30 con 0x7f324c0d1470 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.235+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c140640 con 0x7f324c0d1470 2024-04-26T20:18:22.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.236+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c141710 con 0x7f324c0d1470 2024-04-26T20:18:22.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.237+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f323c03ccd0 con 0x7f324c0d1470 2024-04-26T20:18:22.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.238+0000 7f3242a00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 0x7f322806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.238+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f323c0aca00 con 0x7f324c0d1470 2024-04-26T20:18:22.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.239+0000 7f3243e00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 0x7f322806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.239+0000 7f3243e00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 0x7f322806eed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f324c14c6d0 tx=0x7f3234009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.243+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f324c141710 con 0x7f324c0d1470 2024-04-26T20:18:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.447+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f324c06fce0 con 0x7f324c0d1470 2024-04-26T20:18:22.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.448+0000 7f3242a00640 1 -- 172.21.15.88:0/3276610034 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7f323c046360 con 0x7f324c0d1470 2024-04-26T20:18:22.449 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-04-26T20:18:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 msgr2=0x7f322806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 0x7f322806eed0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f324c14c6d0 tx=0x7f3234009040 comp rx=0 tx=0).stop 2024-04-26T20:18:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 msgr2=0x7f324c162e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f323c02a770 tx=0x7f323c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 shutdown_connections 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f324c141300 0x7f324c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f322806ca10 0x7f322806eed0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f324c14ce70 0x7f324c140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 --2- 172.21.15.88:0/3276610034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f324c0d1470 0x7f324c162e60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.451+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 >> 172.21.15.88:0/3276610034 conn(0x7f324c06c530 msgr2=0x7f324c06de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.452+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 shutdown_connections 2024-04-26T20:18:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.452+0000 7f324be00640 1 -- 172.21.15.88:0/3276610034 wait complete. 2024-04-26T20:18:22.470 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:18:22.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.671+0000 7f31c0800640 1 Processor -- start 2024-04-26T20:18:22.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.671+0000 7f31c0800640 1 -- start start 2024-04-26T20:18:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31c0800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f31b815cb70 0x7f31b806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31c0800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b80ae850 con 0x7f31b815cfd0 2024-04-26T20:18:22.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31c0800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b80aeb30 con 0x7f31b806c770 2024-04-26T20:18:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31c0800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f31b80af460 con 0x7f31b815cb70 2024-04-26T20:18:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31be800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f31b815cfd0 0x7f31b806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47758/0 (socket says 172.21.15.88:47758) 2024-04-26T20:18:22.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.673+0000 7f31be800640 1 -- 172.21.15.88:0/3771851451 learned_addr learned my addr 172.21.15.88:0/3771851451 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883438481 0 0) 0x7f31b80ae850 con 0x7f31b815cfd0 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31a0003520 con 0x7f31b815cfd0 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bd400640 1 --2- 172.21.15.88:0/3771851451 >> v2:172.21.15.132:3300/0 conn(0x7f31b815cb70 0x7f31b806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3986390344 0 0) 0x7f31a0003520 con 0x7f31b815cfd0 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 >> v2:172.21.15.132:3300/0 conn(0x7f31b815cb70 msgr2=0x7f31b806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 --2- 172.21.15.88:0/3771851451 >> v2:172.21.15.132:3300/0 conn(0x7f31b815cb70 0x7f31b806ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 >> v1:172.21.15.132:6789/0 conn(0x7f31b806c770 legacy=0x7f31b806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b80aee10 con 0x7f31b815cfd0 2024-04-26T20:18:22.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f31b40019f0 con 0x7f31b815cfd0 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 >> v1:172.21.15.88:6789/0 conn(0x7f31b815cfd0 legacy=0x7f31b806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 0x7f31a0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a0004800 0x7f31a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31a0005210 0x7f31a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31b80af460 con 0x7f31a00039b0 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f31a000c140 con 0x7f31a0004800 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f31a000c340 con 0x7f31a0005210 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.674+0000 7f31bde00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 0x7f31a0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31be800640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31a0005210 0x7f31a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31bde00640 1 -- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31a0005210 msgr2=0x7f31a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31bde00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31a0005210 0x7f31a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31bde00640 1 -- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a0004800 msgr2=0x7f31a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31bde00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a0004800 0x7f31a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31bde00640 1 -- 172.21.15.88:0/3771851451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a0003520 con 0x7f31a00039b0 2024-04-26T20:18:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31be800640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31a0005210 0x7f31a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.675+0000 7f31be800640 1 -- 172.21.15.88:0/3771851451 reap_dead start 2024-04-26T20:18:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.676+0000 7f31bde00640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 0x7f31a0003df0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31ac001750 tx=0x7f31a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1c760606a0e5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.676+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac001370 con 0x7f31a00039b0 2024-04-26T20:18:22.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.676+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0052d0 con 0x7f31a00039b0 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.676+0000 7f31bca00640 1 -- 172.21.15.88:0/3771851451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac02abc0 con 0x7f31a00039b0 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 -- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 msgr2=0x7f31a0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 0x7f31a0003df0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31ac001750 tx=0x7f31a0005830 comp rx=0 tx=0).stop 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 -- 172.21.15.88:0/3771851451 shutdown_connections 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 --2- 172.21.15.88:0/3771851451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31a00039b0 0x7f31a0003df0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 -- 172.21.15.88:0/3771851451 >> 172.21.15.88:0/3771851451 conn(0x7f31b80b2d50 msgr2=0x7f31b80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 -- 172.21.15.88:0/3771851451 shutdown_connections 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.677+0000 7f31c0800640 1 -- 172.21.15.88:0/3771851451 wait complete. 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.679+0000 7f31c0800640 1 Processor -- start 2024-04-26T20:18:22.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.679+0000 7f31c0800640 1 -- start start 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a00039b0 0x7f31b80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31b81f0e70 0x7f31b80cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31b815c780 con 0x7f31b8154210 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f31b815c600 con 0x7f31a00039b0 2024-04-26T20:18:22.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31c0800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f31b80af8f0 con 0x7f31b81f0e70 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31bd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31bd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44296/0 (socket says 172.21.15.88:44296) 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.680+0000 7f31bd400640 1 -- 172.21.15.88:0/1369756589 learned_addr learned my addr 172.21.15.88:0/1369756589 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 -- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31b81f0e70 msgr2=0x7f31b80cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31b81f0e70 0x7f31b80cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 -- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a00039b0 msgr2=0x7f31b80cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a00039b0 0x7f31b80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 -- 172.21.15.88:0/1369756589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b8152670 con 0x7f31b8154210 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31bd400640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f31a800d800 tx=0x7f31a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800b6c0 con 0x7f31b8154210 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a8019070 con 0x7f31b8154210 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800baa0 con 0x7f31b8154210 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b80ce590 con 0x7f31b8154210 2024-04-26T20:18:22.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.681+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b8152910 con 0x7f31b8154210 2024-04-26T20:18:22.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.683+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b8069540 con 0x7f31b8154210 2024-04-26T20:18:22.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.684+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f31a800bc40 con 0x7f31b8154210 2024-04-26T20:18:22.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.685+0000 7f31b2000640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 0x7f319c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.685+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f31a808c210 con 0x7f31b8154210 2024-04-26T20:18:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.685+0000 7f31bde00640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 0x7f319c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.686+0000 7f31bde00640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 0x7f319c06eea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31ac032000 tx=0x7f31ac02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:22.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.690+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31a805a450 con 0x7f31b8154210 2024-04-26T20:18:22.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.928+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31b80d0680 con 0x7f31b8154210 2024-04-26T20:18:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.928+0000 7f31b2000640 1 -- 172.21.15.88:0/1369756589 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5186 (secure 0 0 0) 0x7f31a8059d50 con 0x7f31b8154210 2024-04-26T20:18:22.929 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:22.929 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:18:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 msgr2=0x7f319c06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 0x7f319c06eea0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31ac032000 tx=0x7f31ac02c000 comp rx=0 tx=0).stop 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 msgr2=0x7f31b814e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f31a800d800 tx=0x7f31a800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 shutdown_connections 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f319c06c9e0 0x7f319c06eea0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f31b81f0e70 0x7f31b80cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31b8154210 0x7f31b814e0b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 --2- 172.21.15.88:0/1369756589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31a00039b0 0x7f31b80cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 >> 172.21.15.88:0/1369756589 conn(0x7f31b80b2d50 msgr2=0x7f31b815d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 shutdown_connections 2024-04-26T20:18:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:22.931+0000 7f31c0800640 1 -- 172.21.15.88:0/1369756589 wait complete. 2024-04-26T20:18:22.950 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:23.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.146+0000 7fae8ee00640 1 Processor -- start 2024-04-26T20:18:23.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.146+0000 7fae8ee00640 1 -- start start 2024-04-26T20:18:23.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8ee00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae8815f150 0x7fae88071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8ee00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae8806ca60 0x7fae880b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8ee00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae88073a90 con 0x7fae8806fa50 2024-04-26T20:18:23.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8ee00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fae880b1060 con 0x7fae8815f150 2024-04-26T20:18:23.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8ee00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fae8806f5c0 con 0x7fae8806ca60 2024-04-26T20:18:23.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8da00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae8815f150 0x7fae88071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8da00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fae8815f150 0x7fae88071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44298/0 (socket says 172.21.15.88:44298) 2024-04-26T20:18:23.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.148+0000 7fae8da00640 1 -- 172.21.15.88:0/1589892467 learned_addr learned my addr 172.21.15.88:0/1589892467 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:23.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.149+0000 7fae8da00640 1 -- 172.21.15.88:0/1589892467 >> v2:172.21.15.132:3300/0 conn(0x7fae8806ca60 msgr2=0x7fae880b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:23.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.149+0000 7fae8da00640 1 --2- 172.21.15.88:0/1589892467 >> v2:172.21.15.132:3300/0 conn(0x7fae8806ca60 0x7fae880b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.149+0000 7fae8da00640 1 -- 172.21.15.88:0/1589892467 >> v1:172.21.15.132:6789/0 conn(0x7fae8806fa50 legacy=0x7fae8806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:23.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.149+0000 7fae8da00640 1 -- 172.21.15.88:0/1589892467 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae88073d70 con 0x7fae8815f150 2024-04-26T20:18:23.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.150+0000 7fae8da00640 1 --2- 172.21.15.88:0/1589892467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae88071590 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fae880d1650 tx=0x7fae7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0e189966ad1d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.150+0000 7fae87e00640 1 -- 172.21.15.88:0/1589892467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c037070 con 0x7fae8815f150 2024-04-26T20:18:23.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.150+0000 7fae87e00640 1 -- 172.21.15.88:0/1589892467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7c032440 con 0x7fae8815f150 2024-04-26T20:18:23.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.150+0000 7fae87e00640 1 -- 172.21.15.88:0/1589892467 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c032810 con 0x7fae8815f150 2024-04-26T20:18:23.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 -- 172.21.15.88:0/1589892467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 msgr2=0x7fae88071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 --2- 172.21.15.88:0/1589892467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae88071590 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fae880d1650 tx=0x7fae7c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 -- 172.21.15.88:0/1589892467 shutdown_connections 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 --2- 172.21.15.88:0/1589892467 >> v2:172.21.15.132:3300/0 conn(0x7fae8806ca60 0x7fae880b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 --2- 172.21.15.88:0/1589892467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae88071590 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 -- 172.21.15.88:0/1589892467 >> 172.21.15.88:0/1589892467 conn(0x7fae880b38a0 msgr2=0x7fae880b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 -- 172.21.15.88:0/1589892467 shutdown_connections 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.151+0000 7fae8ee00640 1 -- 172.21.15.88:0/1589892467 wait complete. 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.153+0000 7fae8ee00640 1 Processor -- start 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.153+0000 7fae8ee00640 1 -- start start 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae8806ca60 0x7fae88155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae8806fa50 0x7fae881f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae8806f250 con 0x7fae8815f150 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fae8806feb0 con 0x7fae8806fa50 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8ee00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fae880b53d0 con 0x7fae8806ca60 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44306/0 (socket says 172.21.15.88:44306) 2024-04-26T20:18:23.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 -- 172.21.15.88:0/2358658703 learned_addr learned my addr 172.21.15.88:0/2358658703 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8da00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae8806ca60 0x7fae88155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8d000640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae8806fa50 0x7fae881f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 -- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae8806ca60 msgr2=0x7fae88155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae8806ca60 0x7fae88155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 -- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae8806fa50 msgr2=0x7fae881f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae8806fa50 0x7fae881f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.154+0000 7fae8e400640 1 -- 172.21.15.88:0/2358658703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae88161e90 con 0x7fae8815f150 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae8e400640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ba10 tx=0x7fae8000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80019070 con 0x7fae8815f150 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8000cb60 con 0x7fae8815f150 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80014440 con 0x7fae8815f150 2024-04-26T20:18:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae88155be0 con 0x7fae8815f150 2024-04-26T20:18:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.155+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae881561a0 con 0x7fae8815f150 2024-04-26T20:18:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.156+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88069ad0 con 0x7fae8815f150 2024-04-26T20:18:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.157+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fae8001c070 con 0x7fae8815f150 2024-04-26T20:18:23.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.158+0000 7fae85600640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 0x7fae6806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.159+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fae8008b9c0 con 0x7fae8815f150 2024-04-26T20:18:23.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.159+0000 7fae8da00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 0x7fae6806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.159+0000 7fae8da00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 0x7fae6806eed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae7c034000 tx=0x7fae7c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.163+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae80059c90 con 0x7fae8815f150 2024-04-26T20:18:23.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.364+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae880708c0 con 0x7fae8815f150 2024-04-26T20:18:23.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.367+0000 7fae85600640 1 -- 172.21.15.88:0/2358658703 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16754 (secure 0 0 0) 0x7fae80059590 con 0x7fae8815f150 2024-04-26T20:18:23.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:23.368 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:03:29.112906+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:23.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.369+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 msgr2=0x7fae6806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.369+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 0x7fae6806eed0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fae7c034000 tx=0x7fae7c02e000 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.369+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 msgr2=0x7fae8815f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.369+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fae8000ba10 tx=0x7fae8000bed0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 shutdown_connections 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae8815f150 0x7fae8815f530 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae8806fa50 0x7fae881f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae6806ca10 0x7fae6806eed0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 --2- 172.21.15.88:0/2358658703 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae8806ca60 0x7fae88155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 >> 172.21.15.88:0/2358658703 conn(0x7fae880b38a0 msgr2=0x7fae880b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 shutdown_connections 2024-04-26T20:18:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.370+0000 7fae8ee00640 1 -- 172.21.15.88:0/2358658703 wait complete. 2024-04-26T20:18:23.389 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:18:23.389 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:18:23.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.575+0000 7fa97e400640 1 Processor -- start 2024-04-26T20:18:23.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.575+0000 7fa97e400640 1 -- start start 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.576+0000 7fa97e400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa97815cdf0 0x7fa9780add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.576+0000 7fa97e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa9780b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.576+0000 7fa97e400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9780af9a0 con 0x7fa9780cd650 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa97e400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fa9780ae2c0 con 0x7fa97815cdf0 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa97e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780b05b0 con 0x7fa97806ce80 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa97815cdf0 0x7fa9780add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fa97815cdf0 0x7fa9780add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44312/0 (socket says 172.21.15.88:44312) 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 -- 172.21.15.88:0/4074532530 learned_addr learned my addr 172.21.15.88:0/4074532530 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:23.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 -- 172.21.15.88:0/4074532530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 msgr2=0x7fa9780b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:23.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977e00640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa9780b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa9780b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 -- 172.21.15.88:0/4074532530 >> v1:172.21.15.132:6789/0 conn(0x7fa9780cd650 legacy=0x7fa97806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:23.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977400640 1 -- 172.21.15.88:0/4074532530 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9780afc80 con 0x7fa97815cdf0 2024-04-26T20:18:23.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.577+0000 7fa977e00640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa9780b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.578+0000 7fa977400640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa97815cdf0 0x7fa9780add80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa968009870 tx=0x7fa96802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1547bc045bf2fe48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa976a00640 1 -- 172.21.15.88:0/4074532530 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968037070 con 0x7fa97815cdf0 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa976a00640 1 -- 172.21.15.88:0/4074532530 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa968032440 con 0x7fa97815cdf0 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa976a00640 1 -- 172.21.15.88:0/4074532530 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968032810 con 0x7fa97815cdf0 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 -- 172.21.15.88:0/4074532530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa97815cdf0 msgr2=0x7fa9780add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa97815cdf0 0x7fa9780add80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa968009870 tx=0x7fa96802a210 comp rx=0 tx=0).stop 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 -- 172.21.15.88:0/4074532530 shutdown_connections 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa9780b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 --2- 172.21.15.88:0/4074532530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa97815cdf0 0x7fa9780add80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 -- 172.21.15.88:0/4074532530 >> 172.21.15.88:0/4074532530 conn(0x7fa9780b2d50 msgr2=0x7fa9780b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 -- 172.21.15.88:0/4074532530 shutdown_connections 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.579+0000 7fa97e400640 1 -- 172.21.15.88:0/4074532530 wait complete. 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.581+0000 7fa97e400640 1 Processor -- start 2024-04-26T20:18:23.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.581+0000 7fa97e400640 1 -- start start 2024-04-26T20:18:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa97809dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa97815cdf0 0x7fa97809c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa97815d1d0 con 0x7fa9780cd650 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780b0240 con 0x7fa97815cdf0 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa97e400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa97815d350 con 0x7fa97806ce80 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa96fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa96fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44318/0 (socket says 172.21.15.88:44318) 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.582+0000 7fa96fe00640 1 -- 172.21.15.88:0/1746194857 learned_addr learned my addr 172.21.15.88:0/1746194857 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa977400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa97809dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 -- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 msgr2=0x7fa97809dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa97809dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 -- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa97815cdf0 msgr2=0x7fa97809c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa977e00640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa97815cdf0 0x7fa97809c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa97815cdf0 0x7fa97809c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 -- 172.21.15.88:0/1746194857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97806f520 con 0x7fa9780cd650 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa977400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa97809dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:23.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96fe00640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c430 tx=0x7fa96000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960011790 con 0x7fa9780cd650 2024-04-26T20:18:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa960017070 con 0x7fa9780cd650 2024-04-26T20:18:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960011b70 con 0x7fa9780cd650 2024-04-26T20:18:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.583+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97809bbd0 con 0x7fa9780cd650 2024-04-26T20:18:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.584+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97806f1b0 con 0x7fa9780cd650 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.584+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978069540 con 0x7fa9780cd650 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.585+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fa960011d10 con 0x7fa9780cd650 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.586+0000 7fa96f400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 0x7fa95406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.586+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fa96008bc60 con 0x7fa9780cd650 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.586+0000 7fa977400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 0x7fa95406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.587+0000 7fa977400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 0x7fa95406eed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa968034000 tx=0x7fa96802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:23.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.591+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa960059ea0 con 0x7fa9780cd650 2024-04-26T20:18:23.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.832+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9780b5740 con 0x7fa9780cd650 2024-04-26T20:18:23.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.833+0000 7fa96f400640 1 -- 172.21.15.88:0/1746194857 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5186 (secure 0 0 0) 0x7fa9600597a0 con 0x7fa9780cd650 2024-04-26T20:18:23.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:23.834 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:18:23.835 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.835+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 msgr2=0x7fa95406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.835+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 0x7fa95406eed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa968034000 tx=0x7fa96802e000 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.835+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 msgr2=0x7fa97806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.835+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c430 tx=0x7fa96000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 shutdown_connections 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa97815cdf0 0x7fa97809c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9780cd650 0x7fa97806fb40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa95406ca10 0x7fa95406eed0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 --2- 172.21.15.88:0/1746194857 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa97806ce80 0x7fa97809dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:23.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 >> 172.21.15.88:0/1746194857 conn(0x7fa9780b2d50 msgr2=0x7fa9780b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:23.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 shutdown_connections 2024-04-26T20:18:23.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:23.836+0000 7fa97e400640 1 -- 172.21.15.88:0/1746194857 wait complete. 2024-04-26T20:18:23.858 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:18:24.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.055+0000 7fae35e00640 1 Processor -- start 2024-04-26T20:18:24.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.055+0000 7fae35e00640 1 -- start start 2024-04-26T20:18:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.057+0000 7fae35e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae3015cdf0 0x7fae300add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.057+0000 7fae35e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae300cd650 0x7fae3006c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.057+0000 7fae35e00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae300af9a0 con 0x7fae300b4f50 2024-04-26T20:18:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.057+0000 7fae35e00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fae300ae2c0 con 0x7fae3015cdf0 2024-04-26T20:18:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.057+0000 7fae35e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fae300b05b0 con 0x7fae300cd650 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.058+0000 7fae2e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae300cd650 0x7fae3006c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.058+0000 7fae2f400640 1 --1- >> v1:172.21.15.132:6789/0 conn(0x7fae300b4f50 0x7fae30072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.132:6789/0 says I am v1:172.21.15.88:52996/0 (socket says 172.21.15.88:52996) 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.058+0000 7fae2ea00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fae3015cdf0 0x7fae300add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.058+0000 7fae2f400640 1 -- 172.21.15.88:0/2754035080 learned_addr learned my addr 172.21.15.88:0/2754035080 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.059+0000 7fae2d600640 1 -- 172.21.15.88:0/2754035080 <== mon.1 v1:172.21.15.132:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890161922 0 0) 0x7fae300af9a0 con 0x7fae300b4f50 2024-04-26T20:18:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.059+0000 7fae2ea00640 1 -- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae300cd650 msgr2=0x7fae3006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.059+0000 7fae2ea00640 1 --2- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae300cd650 0x7fae3006c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.059+0000 7fae2ea00640 1 -- 172.21.15.88:0/2754035080 >> v1:172.21.15.132:6789/0 conn(0x7fae300b4f50 legacy=0x7fae30072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.059+0000 7fae2ea00640 1 -- 172.21.15.88:0/2754035080 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae300afc80 con 0x7fae3015cdf0 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.061+0000 7fae2ea00640 1 --2- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae3015cdf0 0x7fae300add80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fae200097a0 tx=0x7fae2002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d95cf2f7685f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.061+0000 7fae2d600640 1 -- 172.21.15.88:0/2754035080 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20037070 con 0x7fae3015cdf0 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae2d600640 1 -- 172.21.15.88:0/2754035080 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20032440 con 0x7fae3015cdf0 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae2d600640 1 -- 172.21.15.88:0/2754035080 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20032810 con 0x7fae3015cdf0 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 -- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae3015cdf0 msgr2=0x7fae300add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 --2- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae3015cdf0 0x7fae300add80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fae200097a0 tx=0x7fae2002a180 comp rx=0 tx=0).stop 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 -- 172.21.15.88:0/2754035080 shutdown_connections 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 --2- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae300cd650 0x7fae3006c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 --2- 172.21.15.88:0/2754035080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae3015cdf0 0x7fae300add80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.062+0000 7fae35e00640 1 -- 172.21.15.88:0/2754035080 >> 172.21.15.88:0/2754035080 conn(0x7fae300b2d50 msgr2=0x7fae300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.063+0000 7fae35e00640 1 -- 172.21.15.88:0/2754035080 shutdown_connections 2024-04-26T20:18:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.063+0000 7fae35e00640 1 -- 172.21.15.88:0/2754035080 wait complete. 2024-04-26T20:18:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.064+0000 7fae35e00640 1 Processor -- start 2024-04-26T20:18:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.064+0000 7fae35e00640 1 -- start start 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae300b4f50 0x7fae3014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae3015cdf0 0x7fae3009d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae3015d1d0 con 0x7fae300cd650 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fae300b0240 con 0x7fae300b4f50 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.065+0000 7fae35e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fae3015d350 con 0x7fae3015cdf0 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44322/0 (socket says 172.21.15.88:44322) 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 -- 172.21.15.88:0/3635707307 learned_addr learned my addr 172.21.15.88:0/3635707307 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2ea00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae300b4f50 0x7fae3014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2f400640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae3015cdf0 0x7fae3009d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 -- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae3015cdf0 msgr2=0x7fae3009d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae3015cdf0 0x7fae3009d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 -- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae300b4f50 msgr2=0x7fae3014e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae300b4f50 0x7fae3014e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.066+0000 7fae2e000640 1 -- 172.21.15.88:0/3635707307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3014ec40 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae2e000640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e9e0 tx=0x7fae1c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c017070 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c00c9d0 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00cdc0 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3015d800 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.067+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3009db10 con 0x7fae300cd650 2024-04-26T20:18:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.068+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30069540 con 0x7fae300cd650 2024-04-26T20:18:24.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.069+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fae1c01a070 con 0x7fae300cd650 2024-04-26T20:18:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.070+0000 7fae27400640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 0x7fae1006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.070+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fae1c08bde0 con 0x7fae300cd650 2024-04-26T20:18:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.070+0000 7fae2ea00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 0x7fae1006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.071+0000 7fae2ea00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 0x7fae1006eed0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.074+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae1c05a020 con 0x7fae300cd650 2024-04-26T20:18:24.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.313+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae3009dce0 con 0x7fae300cd650 2024-04-26T20:18:24.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.315+0000 7fae27400640 1 -- 172.21.15.88:0/3635707307 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5186 (secure 0 0 0) 0x7fae1c059920 con 0x7fae300cd650 2024-04-26T20:18:24.315 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:24.315 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:02:43.647939+0000","modified":"2024-04-26T20:02:46.699316+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6835/1355496095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":1355496095},{"type":"v1","addr":"172.21.15.132:6835","nonce":1355496095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-04-26T20:18:24.316 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 msgr2=0x7fae1006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 0x7fae1006eed0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).stop 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 msgr2=0x7fae3009ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e9e0 tx=0x7fae1c00eea0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 shutdown_connections 2024-04-26T20:18:24.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fae1006ca10 0x7fae1006eed0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fae3015cdf0 0x7fae3009d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae300cd650 0x7fae3009ccc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 --2- 172.21.15.88:0/3635707307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae300b4f50 0x7fae3014e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 >> 172.21.15.88:0/3635707307 conn(0x7fae300b2d50 msgr2=0x7fae300b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.317+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 shutdown_connections 2024-04-26T20:18:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.318+0000 7fae35e00640 1 -- 172.21.15.88:0/3635707307 wait complete. 2024-04-26T20:18:24.336 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.527+0000 7ff0b7400640 1 Processor -- start 2024-04-26T20:18:24.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.527+0000 7ff0b7400640 1 -- start start 2024-04-26T20:18:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b7400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7ff0b006ce80 0x7ff0b00add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b7400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 0x7ff0b00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b7400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b00af9a0 con 0x7ff0b015cdf0 2024-04-26T20:18:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b7400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b00ae2c0 con 0x7ff0b006ce80 2024-04-26T20:18:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b7400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b00b05b0 con 0x7ff0b00cd650 2024-04-26T20:18:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b4a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7ff0b015cdf0 0x7ff0b006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47766/0 (socket says 172.21.15.88:47766) 2024-04-26T20:18:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0b4a00640 1 -- 172.21.15.88:0/117159594 learned_addr learned my addr 172.21.15.88:0/117159594 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:24.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.529+0000 7ff0afe00640 1 --2- 172.21.15.88:0/117159594 >> v2:172.21.15.88:3300/0 conn(0x7ff0b006ce80 0x7ff0b00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0af400640 1 -- 172.21.15.88:0/117159594 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671569348 0 0) 0x7ff0b00af9a0 con 0x7ff0b015cdf0 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0af400640 1 -- 172.21.15.88:0/117159594 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff094003520 con 0x7ff0b015cdf0 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0afe00640 1 -- 172.21.15.88:0/117159594 >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 msgr2=0x7ff0b00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0b5400640 1 --2- 172.21.15.88:0/117159594 >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 0x7ff0b00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0afe00640 1 --2- 172.21.15.88:0/117159594 >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 0x7ff0b00b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0afe00640 1 -- 172.21.15.88:0/117159594 >> v1:172.21.15.88:6789/0 conn(0x7ff0b015cdf0 legacy=0x7ff0b006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0afe00640 1 -- 172.21.15.88:0/117159594 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b00afc80 con 0x7ff0b006ce80 2024-04-26T20:18:24.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.530+0000 7ff0b5400640 1 --2- 172.21.15.88:0/117159594 >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 0x7ff0b00b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.531+0000 7ff0afe00640 1 --2- 172.21.15.88:0/117159594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b00add80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00b0520 tx=0x7ff09c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57502a8942f20868 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.531+0000 7ff0af400640 1 -- 172.21.15.88:0/117159594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c037070 con 0x7ff0b006ce80 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.531+0000 7ff0af400640 1 -- 172.21.15.88:0/117159594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c032440 con 0x7ff0b006ce80 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.531+0000 7ff0af400640 1 -- 172.21.15.88:0/117159594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c032810 con 0x7ff0b006ce80 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 -- 172.21.15.88:0/117159594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 msgr2=0x7ff0b00add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 --2- 172.21.15.88:0/117159594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b00add80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00b0520 tx=0x7ff09c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 -- 172.21.15.88:0/117159594 shutdown_connections 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 --2- 172.21.15.88:0/117159594 >> v2:172.21.15.132:3300/0 conn(0x7ff0b00cd650 0x7ff0b00b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 --2- 172.21.15.88:0/117159594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b00add80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 -- 172.21.15.88:0/117159594 >> 172.21.15.88:0/117159594 conn(0x7ff0b00b2d50 msgr2=0x7ff0b00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 -- 172.21.15.88:0/117159594 shutdown_connections 2024-04-26T20:18:24.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.532+0000 7ff0b7400640 1 -- 172.21.15.88:0/117159594 wait complete. 2024-04-26T20:18:24.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.534+0000 7ff0b7400640 1 Processor -- start 2024-04-26T20:18:24.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.534+0000 7ff0b7400640 1 -- start start 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b00cd650 0x7ff0b015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b015cdf0 0x7ff0b0146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b015d1d0 con 0x7ff0b006ce80 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b00b0240 con 0x7ff0b015cdf0 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b7400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b015d350 con 0x7ff0b00cd650 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44338/0 (socket says 172.21.15.88:44338) 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 -- 172.21.15.88:0/2087700676 learned_addr learned my addr 172.21.15.88:0/2087700676 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:24.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0afe00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b00cd650 0x7ff0b015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 -- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b00cd650 msgr2=0x7ff0b015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b00cd650 0x7ff0b015fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 -- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b015cdf0 msgr2=0x7ff0b0146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b015cdf0 0x7ff0b0146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.535+0000 7ff0b4a00640 1 -- 172.21.15.88:0/2087700676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b0149c70 con 0x7ff0b006ce80 2024-04-26T20:18:24.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0b4a00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c390 tx=0x7ff0a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0017070 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a0005ce0 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00072f0 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b0149ee0 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.536+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b014a4a0 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.537+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0069540 con 0x7ff0b006ce80 2024-04-26T20:18:24.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.538+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7ff0a001a070 con 0x7ff0b006ce80 2024-04-26T20:18:24.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.539+0000 7ff0acc00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 0x7ff09006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.539+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7ff0a008b140 con 0x7ff0b006ce80 2024-04-26T20:18:24.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.539+0000 7ff0afe00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 0x7ff09006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.540+0000 7ff0afe00640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 0x7ff09006eed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff09c034000 tx=0x7ff09c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.543+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0a0059410 con 0x7ff0b006ce80 2024-04-26T20:18:24.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.741+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0b0159cb0 con 0x7ff0b006ce80 2024-04-26T20:18:24.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.743+0000 7ff0acc00640 1 -- 172.21.15.88:0/2087700676 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16754 (secure 0 0 0) 0x7ff0a0058d10 con 0x7ff0b006ce80 2024-04-26T20:18:24.744 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:24.745 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:03:29.112906+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:02:41.445900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-26T20:02:42.613083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:24.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 msgr2=0x7ff09006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 0x7ff09006eed0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff09c034000 tx=0x7ff09c02e000 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 msgr2=0x7ff0b0149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c390 tx=0x7ff0a000c850 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 shutdown_connections 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0b015cdf0 0x7ff0b0146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff09006ca10 0x7ff09006eed0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff0b00cd650 0x7ff0b015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 --2- 172.21.15.88:0/2087700676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0b006ce80 0x7ff0b0149730 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 >> 172.21.15.88:0/2087700676 conn(0x7ff0b00b2d50 msgr2=0x7ff0b00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 shutdown_connections 2024-04-26T20:18:24.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.746+0000 7ff0b7400640 1 -- 172.21.15.88:0/2087700676 wait complete. 2024-04-26T20:18:24.764 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:18:24.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.956+0000 7f2af0400640 1 Processor -- start 2024-04-26T20:18:24.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.957+0000 7f2af0400640 1 -- start start 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.958+0000 7f2af0400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ae80ade30 0x7f2ae806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.958+0000 7f2af0400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f2ae80737c0 0x7f2ae81663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.958+0000 7f2af0400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae80ae5f0 con 0x7f2ae806fb50 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.958+0000 7f2af0400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae8072010 con 0x7f2ae80ade30 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.958+0000 7f2af0400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f2ae80722f0 con 0x7f2ae80737c0 2024-04-26T20:18:24.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ae80ade30 0x7f2ae806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f2ae80ade30 0x7f2ae806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44350/0 (socket says 172.21.15.88:44350) 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 -- 172.21.15.88:0/3167558802 learned_addr learned my addr 172.21.15.88:0/3167558802 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 -- 172.21.15.88:0/3167558802 >> v2:172.21.15.132:3300/0 conn(0x7f2ae80737c0 msgr2=0x7f2ae81663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 --2- 172.21.15.88:0/3167558802 >> v2:172.21.15.132:3300/0 conn(0x7f2ae80737c0 0x7f2ae81663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 -- 172.21.15.88:0/3167558802 >> v1:172.21.15.132:6789/0 conn(0x7f2ae806fb50 legacy=0x7f2ae80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:24.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.959+0000 7f2aeda00640 1 -- 172.21.15.88:0/3167558802 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae80725d0 con 0x7f2ae80ade30 2024-04-26T20:18:24.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.960+0000 7f2aeda00640 1 --2- 172.21.15.88:0/3167558802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae80ade30 0x7f2ae806c770 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b4350 tx=0x7f2adc02c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da02e90b021b6e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2ae7e00640 1 -- 172.21.15.88:0/3167558802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc038070 con 0x7f2ae80ade30 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2ae7e00640 1 -- 172.21.15.88:0/3167558802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc031710 con 0x7f2ae80ade30 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2ae7e00640 1 -- 172.21.15.88:0/3167558802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc031ae0 con 0x7f2ae80ade30 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 -- 172.21.15.88:0/3167558802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae80ade30 msgr2=0x7f2ae806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 --2- 172.21.15.88:0/3167558802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae80ade30 0x7f2ae806c770 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b4350 tx=0x7f2adc02c440 comp rx=0 tx=0).stop 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 -- 172.21.15.88:0/3167558802 shutdown_connections 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 --2- 172.21.15.88:0/3167558802 >> v2:172.21.15.132:3300/0 conn(0x7f2ae80737c0 0x7f2ae81663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 --2- 172.21.15.88:0/3167558802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae80ade30 0x7f2ae806c770 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 -- 172.21.15.88:0/3167558802 >> 172.21.15.88:0/3167558802 conn(0x7f2ae80b2d50 msgr2=0x7f2ae80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 -- 172.21.15.88:0/3167558802 shutdown_connections 2024-04-26T20:18:24.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.961+0000 7f2af0400640 1 -- 172.21.15.88:0/3167558802 wait complete. 2024-04-26T20:18:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.963+0000 7f2af0400640 1 Processor -- start 2024-04-26T20:18:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.963+0000 7f2af0400640 1 -- start start 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ae80737c0 0x7f2ae80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ae80ade30 0x7f2ae80b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80ae210 con 0x7f2ae806fb50 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae8072b90 con 0x7f2ae80ade30 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2af0400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae80ae390 con 0x7f2ae80737c0 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.964+0000 7f2aeda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44366/0 (socket says 172.21.15.88:44366) 2024-04-26T20:18:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 -- 172.21.15.88:0/1011976350 learned_addr learned my addr 172.21.15.88:0/1011976350 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:24.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 -- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ae80737c0 msgr2=0x7f2ae80cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ae80737c0 0x7f2ae80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 -- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ae80ade30 msgr2=0x7f2ae80b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ae80ade30 0x7f2ae80b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 -- 172.21.15.88:0/1011976350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae81560c0 con 0x7f2ae806fb50 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.965+0000 7f2aeda00640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.966+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc042020 con 0x7f2ae806fb50 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.966+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc033830 con 0x7f2ae806fb50 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.966+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc038040 con 0x7f2ae806fb50 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.966+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae80cfdd0 con 0x7f2ae806fb50 2024-04-26T20:18:24.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.966+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae80d0390 con 0x7f2ae806fb50 2024-04-26T20:18:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.967+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae8069540 con 0x7f2ae806fb50 2024-04-26T20:18:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.968+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f2adc045020 con 0x7f2ae806fb50 2024-04-26T20:18:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.969+0000 7f2ae5600640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 0x7f2acc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.969+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f2adc0ac9f0 con 0x7f2ae806fb50 2024-04-26T20:18:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.969+0000 7f2aed000640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 0x7f2acc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:24.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.970+0000 7f2aed000640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 0x7f2acc06eed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ae809dc80 tx=0x7f2ad8006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:24.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:24.973+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2adc04c030 con 0x7f2ae806fb50 2024-04-26T20:18:25.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.195+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2ae80b34b0 con 0x7f2ae806fb50 2024-04-26T20:18:25.196 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-04-26T20:18:25.195+0000 7fad57a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:18:25.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.198+0000 7f2ae5600640 1 -- 172.21.15.88:0/1011976350 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7f2adc07acc0 con 0x7f2ae806fb50 2024-04-26T20:18:25.199 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:18:25.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 msgr2=0x7f2acc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 0x7f2acc06eed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2ae809dc80 tx=0x7f2ad8006ba0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 msgr2=0x7f2ae81506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 shutdown_connections 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ae80ade30 0x7f2ae80b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2acc06ca10 0x7f2acc06eed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2ae80737c0 0x7f2ae80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 --2- 172.21.15.88:0/1011976350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ae806fb50 0x7f2ae81506e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 >> 172.21.15.88:0/1011976350 conn(0x7f2ae80b2d50 msgr2=0x7f2ae815dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:25.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.202+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 shutdown_connections 2024-04-26T20:18:25.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.203+0000 7f2af0400640 1 -- 172.21.15.88:0/1011976350 wait complete. 2024-04-26T20:18:25.222 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:18:25.327 INFO:tasks.ceph.mds.d.smithi132.stdout:starting mds.d at 2024-04-26T20:18:25.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.416+0000 7f21ec800640 1 Processor -- start 2024-04-26T20:18:25.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.417+0000 7f21ec800640 1 -- start start 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21e40d19c0 0x7f21e40d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 0x7f21e4162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f21e40d40d0 con 0x7f21e40d19c0 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f21e40d3830 con 0x7f21e40d1da0 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21ec800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f21e40d3b10 con 0x7f21e40b6170 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.418+0000 7f21e9e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21e40d19c0 0x7f21e40d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f21e40d19c0 0x7f21e40d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44374/0 (socket says 172.21.15.88:44374) 2024-04-26T20:18:25.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 -- 172.21.15.88:0/2580080007 learned_addr learned my addr 172.21.15.88:0/2580080007 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:25.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9400640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 -- 172.21.15.88:0/2580080007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 msgr2=0x7f21e40c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:25.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21ea800640 1 --2- 172.21.15.88:0/2580080007 >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 0x7f21e4162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e40c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 -- 172.21.15.88:0/2580080007 >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 msgr2=0x7f21e4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2580080007 >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 0x7f21e4162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21e9e00640 1 -- 172.21.15.88:0/2580080007 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40d3df0 con 0x7f21e40d19c0 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.419+0000 7f21ea800640 1 --2- 172.21.15.88:0/2580080007 >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 0x7f21e4162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:25.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.420+0000 7f21e9e00640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40d2f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cc160 tx=0x7f21dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf86650862c9b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.420+0000 7f21e8a00640 1 -- 172.21.15.88:0/2580080007 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc036070 con 0x7f21e40d19c0 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21e8a00640 1 -- 172.21.15.88:0/2580080007 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc031600 con 0x7f21e40d19c0 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21e8a00640 1 -- 172.21.15.88:0/2580080007 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0319d0 con 0x7f21e40d19c0 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 -- 172.21.15.88:0/2580080007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 msgr2=0x7f21e40d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40d2f00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cc160 tx=0x7f21dc02a610 comp rx=0 tx=0).stop 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 -- 172.21.15.88:0/2580080007 shutdown_connections 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 --2- 172.21.15.88:0/2580080007 >> v2:172.21.15.132:3300/0 conn(0x7f21e40d1da0 0x7f21e4162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 --2- 172.21.15.88:0/2580080007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40d2f00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 -- 172.21.15.88:0/2580080007 >> 172.21.15.88:0/2580080007 conn(0x7f21e409e460 msgr2=0x7f21e40d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:25.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.421+0000 7f21ec800640 1 -- 172.21.15.88:0/2580080007 shutdown_connections 2024-04-26T20:18:25.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.422+0000 7f21ec800640 1 -- 172.21.15.88:0/2580080007 wait complete. 2024-04-26T20:18:25.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.424+0000 7f21ec800640 1 Processor -- start 2024-04-26T20:18:25.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.424+0000 7f21ec800640 1 -- start start 2024-04-26T20:18:25.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40d1da0 0x7f21e40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40cfc30 con 0x7f21e40d19c0 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f21e40c91b0 con 0x7f21e40d1da0 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21ec800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f21e4162b80 con 0x7f21e40b6170 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44378/0 (socket says 172.21.15.88:44378) 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.425+0000 7f21e9400640 1 -- 172.21.15.88:0/3054033919 learned_addr learned my addr 172.21.15.88:0/3054033919 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9e00640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 -- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 msgr2=0x7f21e415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21ea800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40d1da0 0x7f21e40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 -- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40d1da0 msgr2=0x7f21e40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:25.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40d1da0 0x7f21e40b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:25.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 -- 172.21.15.88:0/3054033919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40c8ea0 con 0x7f21e40d19c0 2024-04-26T20:18:25.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21e9400640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f21d400d830 tx=0x7f21d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4018020 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d40096a0 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4009a70 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.426+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e40b28c0 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.427+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e40b2e80 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.428+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e4069650 con 0x7f21e40d19c0 2024-04-26T20:18:25.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.429+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f21d401b020 con 0x7f21e40d19c0 2024-04-26T20:18:25.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.430+0000 7f21da000640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 0x7f21c806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:25.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.430+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5853+0+0 (secure 0 0 0) 0x7f21d408c3b0 con 0x7f21e40d19c0 2024-04-26T20:18:25.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.430+0000 7f21e9e00640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 0x7f21c806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:25.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.431+0000 7f21e9e00640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 0x7f21c806ef20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f21dc033000 tx=0x7f21dc009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:25.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.434+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21d4022020 con 0x7f21e40d19c0 2024-04-26T20:18:25.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:25.673+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f21e40d26d0 con 0x7f21e40d19c0 2024-04-26T20:18:26.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.209+0000 7f21da000640 1 -- 172.21.15.88:0/3054033919 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f21d4011340 con 0x7f21e40d19c0 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 msgr2=0x7f21c806ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 0x7f21c806ef20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f21dc033000 tx=0x7f21dc009000 comp rx=0 tx=0).stop 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 msgr2=0x7f21e40b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f21d400d830 tx=0x7f21d400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 shutdown_connections 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21e40d1da0 0x7f21e40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e40d19c0 0x7f21e40b1d50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f21c806ca60 0x7f21c806ef20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 --2- 172.21.15.88:0/3054033919 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f21e40b6170 0x7f21e415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 >> 172.21.15.88:0/3054033919 conn(0x7f21e409e460 msgr2=0x7f21e4160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 shutdown_connections 2024-04-26T20:18:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.213+0000 7f21ec800640 1 -- 172.21.15.88:0/3054033919 wait complete. 2024-04-26T20:18:26.237 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:18:26.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.443+0000 7fe2f1600640 1 Processor -- start 2024-04-26T20:18:26.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.444+0000 7fe2f1600640 1 -- start start 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f1600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe2ec071e60 0x7fe2ec0722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f1600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ec072a20 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f1600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ec0cf210 con 0x7fe2ec1459b0 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f1600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ec0b5be0 con 0x7fe2ec071e60 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f0c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe2ec0d0f40 0x7fe2ec0b3a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47770/0 (socket says 172.21.15.88:47770) 2024-04-26T20:18:26.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.445+0000 7fe2f0c00640 1 -- 172.21.15.88:0/2523422745 learned_addr learned my addr 172.21.15.88:0/2523422745 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1944626485 0 0) 0x7fe2ec072a20 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2e3400640 1 --2- 172.21.15.88:0/2523422745 >> v2:172.21.15.132:3300/0 conn(0x7fe2ec071e60 0x7fe2ec0722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2cc003520 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2901244680 0 0) 0x7fe2cc003520 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 >> v2:172.21.15.132:3300/0 conn(0x7fe2ec071e60 msgr2=0x7fe2ec0722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 --2- 172.21.15.88:0/2523422745 >> v2:172.21.15.132:3300/0 conn(0x7fe2ec071e60 0x7fe2ec0722d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 >> v1:172.21.15.132:6789/0 conn(0x7fe2ec1459b0 legacy=0x7fe2ec0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec0cf4f0 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fe2e40019f0 con 0x7fe2ec0d0f40 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 >> v1:172.21.15.88:6789/0 conn(0x7fe2ec0d0f40 legacy=0x7fe2ec0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2cc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2cc0051c0 0x7fe2cc00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc0104d0 con 0x7fe2cc0039b0 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc010680 con 0x7fe2cc0051c0 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.446+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe2cc010880 con 0x7fe2cc004800 2024-04-26T20:18:26.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2cc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2f0c00640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2cc0051c0 0x7fe2cc00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2e3400640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2cc004800 msgr2=0x7fe2cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2cc004800 0x7fe2cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2cc0051c0 msgr2=0x7fe2cc00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2cc0051c0 0x7fe2cc00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2523422745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc003520 con 0x7fe2cc0039b0 2024-04-26T20:18:26.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.447+0000 7fe2f0c00640 1 -- 172.21.15.88:0/2523422745 reap_dead start 2024-04-26T20:18:26.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.449+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2cc003db0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc003d80 tx=0x7fe2cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666cabd537754f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:26.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.449+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc037070 con 0x7fe2cc0039b0 2024-04-26T20:18:26.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.449+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc02a9d0 con 0x7fe2cc0039b0 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.449+0000 7fe2eb400640 1 -- 172.21.15.88:0/2523422745 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc003980 con 0x7fe2cc0039b0 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.449+0000 7fe2f1600640 1 -- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 msgr2=0x7fe2cc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2cc003db0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc003d80 tx=0x7fe2cc005830 comp rx=0 tx=0).stop 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 -- 172.21.15.88:0/2523422745 shutdown_connections 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2523422745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2cc003db0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 -- 172.21.15.88:0/2523422745 >> 172.21.15.88:0/2523422745 conn(0x7fe2ec06c770 msgr2=0x7fe2ec06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 -- 172.21.15.88:0/2523422745 shutdown_connections 2024-04-26T20:18:26.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.450+0000 7fe2f1600640 1 -- 172.21.15.88:0/2523422745 wait complete. 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.451+0000 7fe2f1600640 1 Processor -- start 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 -- start start 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec145210 0x7fe2ec143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec159030 0x7fe2ec142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec145d90 con 0x7fe2cc0039b0 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec072d00 con 0x7fe2ec145210 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.452+0000 7fe2f1600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec145f10 con 0x7fe2ec159030 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44384/0 (socket says 172.21.15.88:44384) 2024-04-26T20:18:26.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2582856559 learned_addr learned my addr 172.21.15.88:0/2582856559 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2e3400640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec145210 0x7fe2ec143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec159030 msgr2=0x7fe2ec142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec159030 0x7fe2ec142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec145210 msgr2=0x7fe2ec143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec145210 0x7fe2ec143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.453+0000 7fe2ebe00640 1 -- 172.21.15.88:0/2582856559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec143ea0 con 0x7fe2cc0039b0 2024-04-26T20:18:26.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2ebe00640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc035040 tx=0x7fe2dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc042020 con 0x7fe2cc0039b0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc038040 con 0x7fe2cc0039b0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc037050 con 0x7fe2cc0039b0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec144170 con 0x7fe2cc0039b0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.454+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec144730 con 0x7fe2cc0039b0 2024-04-26T20:18:26.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.455+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ec0693c0 con 0x7fe2cc0039b0 2024-04-26T20:18:26.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.456+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fe2dc007450 con 0x7fe2cc0039b0 2024-04-26T20:18:26.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.457+0000 7fe2e8c00640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 0x7fe2c806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:26.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.457+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5853+0+0 (secure 0 0 0) 0x7fe2dc0abd50 con 0x7fe2cc0039b0 2024-04-26T20:18:26.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.457+0000 7fe2e3400640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 0x7fe2c806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:26.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.458+0000 7fe2e3400640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 0x7fe2c806eed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec142e50 tx=0x7fe2d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:26.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.461+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2dc04c020 con 0x7fe2cc0039b0 2024-04-26T20:18:26.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:26.663+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2ec159450 con 0x7fe2cc0039b0 2024-04-26T20:18:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.632+0000 7fe2e8c00640 1 -- 172.21.15.88:0/2582856559 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) ==== 181+0+0 (secure 0 0 0) 0x7fe2dc079fd0 con 0x7fe2cc0039b0 2024-04-26T20:18:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:18:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 msgr2=0x7fe2c806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 0x7fe2c806eed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec142e50 tx=0x7fe2d4008040 comp rx=0 tx=0).stop 2024-04-26T20:18:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 msgr2=0x7fe2ec15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc035040 tx=0x7fe2dc0036f0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 shutdown_connections 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe2c806ca10 0x7fe2c806eed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe2ec159030 0x7fe2ec142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2ec145210 0x7fe2ec143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 --2- 172.21.15.88:0/2582856559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2cc0039b0 0x7fe2ec15d880 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 >> 172.21.15.88:0/2582856559 conn(0x7fe2ec06c770 msgr2=0x7fe2ec15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:27.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 shutdown_connections 2024-04-26T20:18:27.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.637+0000 7fe2f1600640 1 -- 172.21.15.88:0/2582856559 wait complete. 2024-04-26T20:18:27.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:18:27.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.853+0000 7fe776600640 1 Processor -- start 2024-04-26T20:18:27.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.854+0000 7fe776600640 1 -- start start 2024-04-26T20:18:27.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe776600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe770142600 0x7fe7700cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe776600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe7700cb6e0 0x7fe77006f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe776600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7700cbc60 con 0x7fe77006fda0 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe776600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe770070210 con 0x7fe7700cb6e0 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe776600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe7700ccd00 con 0x7fe770142600 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe767e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe7700cb6e0 0x7fe77006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.855+0000 7fe775c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe77006fda0 0x7fe77015fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47786/0 (socket says 172.21.15.88:47786) 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe775c00640 1 -- 172.21.15.88:0/1828384917 learned_addr learned my addr 172.21.15.88:0/1828384917 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe775200640 1 --2- 172.21.15.88:0/1828384917 >> v2:172.21.15.132:3300/0 conn(0x7fe770142600 0x7fe7700cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe767e00640 1 -- 172.21.15.88:0/1828384917 >> v2:172.21.15.132:3300/0 conn(0x7fe770142600 msgr2=0x7fe7700cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe767e00640 1 --2- 172.21.15.88:0/1828384917 >> v2:172.21.15.132:3300/0 conn(0x7fe770142600 0x7fe7700cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe767e00640 1 -- 172.21.15.88:0/1828384917 >> v1:172.21.15.88:6789/0 conn(0x7fe77006fda0 legacy=0x7fe77015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe767e00640 1 -- 172.21.15.88:0/1828384917 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7700cbf40 con 0x7fe7700cb6e0 2024-04-26T20:18:27.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.856+0000 7fe76fe00640 1 -- 172.21.15.88:0/1828384917 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 589299459 0 0) 0x7fe7700cbc60 con 0x7fe77006fda0 2024-04-26T20:18:27.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.857+0000 7fe767e00640 1 --2- 172.21.15.88:0/1828384917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7700cb6e0 0x7fe77006f860 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe770069170 tx=0x7fe75802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d350de0e770504f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:27.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.857+0000 7fe76fe00640 1 -- 172.21.15.88:0/1828384917 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758037070 con 0x7fe7700cb6e0 2024-04-26T20:18:27.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe76fe00640 1 -- 172.21.15.88:0/1828384917 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe758032440 con 0x7fe7700cb6e0 2024-04-26T20:18:27.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe76fe00640 1 -- 172.21.15.88:0/1828384917 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75802ae60 con 0x7fe7700cb6e0 2024-04-26T20:18:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 -- 172.21.15.88:0/1828384917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7700cb6e0 msgr2=0x7fe77006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 --2- 172.21.15.88:0/1828384917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7700cb6e0 0x7fe77006f860 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe770069170 tx=0x7fe75802a110 comp rx=0 tx=0).stop 2024-04-26T20:18:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 -- 172.21.15.88:0/1828384917 shutdown_connections 2024-04-26T20:18:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 --2- 172.21.15.88:0/1828384917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7700cb6e0 0x7fe77006f860 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 --2- 172.21.15.88:0/1828384917 >> v2:172.21.15.132:3300/0 conn(0x7fe770142600 0x7fe7700cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 -- 172.21.15.88:0/1828384917 >> 172.21.15.88:0/1828384917 conn(0x7fe7700d0b50 msgr2=0x7fe7700d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:27.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 -- 172.21.15.88:0/1828384917 shutdown_connections 2024-04-26T20:18:27.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.858+0000 7fe776600640 1 -- 172.21.15.88:0/1828384917 wait complete. 2024-04-26T20:18:27.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.860+0000 7fe776600640 1 Processor -- start 2024-04-26T20:18:27.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.860+0000 7fe776600640 1 -- start start 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77006fda0 0x7fe77013c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 0x7fe7700b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7700cc7e0 con 0x7fe770142600 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe7700cc500 con 0x7fe7700cb6e0 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe776600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe770072de0 con 0x7fe77006fda0 2024-04-26T20:18:27.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.861+0000 7fe775c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44398/0 (socket says 172.21.15.88:44398) 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 -- 172.21.15.88:0/3710528715 learned_addr learned my addr 172.21.15.88:0/3710528715 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe767e00640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 0x7fe7700b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775200640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77006fda0 0x7fe77013c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 -- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77006fda0 msgr2=0x7fe77013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77006fda0 0x7fe77013c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 -- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 msgr2=0x7fe7700b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 0x7fe7700b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 -- 172.21.15.88:0/3710528715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770158c40 con 0x7fe770142600 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe767e00640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 0x7fe7700b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.862+0000 7fe775c00640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe76800c730 tx=0x7fe76800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.863+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768018070 con 0x7fe770142600 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.863+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe76800ce40 con 0x7fe770142600 2024-04-26T20:18:27.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.863+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76800a860 con 0x7fe770142600 2024-04-26T20:18:27.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.863+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77013cf00 con 0x7fe770142600 2024-04-26T20:18:27.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.863+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77013d4c0 con 0x7fe770142600 2024-04-26T20:18:27.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.864+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77006d1f0 con 0x7fe770142600 2024-04-26T20:18:27.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.865+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fe76801b070 con 0x7fe770142600 2024-04-26T20:18:27.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.866+0000 7fe76d600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 0x7fe74c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:27.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.866+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5415+0+0 (secure 0 0 0) 0x7fe76808b1c0 con 0x7fe770142600 2024-04-26T20:18:27.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.866+0000 7fe775200640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 0x7fe74c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:27.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.867+0000 7fe775200640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 0x7fe74c06eed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe760006fd0 tx=0x7fe760008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:27.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:27.871+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe77006d1f0 con 0x7fe770142600 2024-04-26T20:18:28.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.069+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe77006e3a0 con 0x7fe770142600 2024-04-26T20:18:28.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.636+0000 7fe76d600640 1 -- 172.21.15.88:0/3710528715 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) ==== 169+0+0 (secure 0 0 0) 0x7fe7680595f0 con 0x7fe770142600 2024-04-26T20:18:28.637 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-04-26T20:18:28.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 msgr2=0x7fe74c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 0x7fe74c06eed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe760006fd0 tx=0x7fe760008040 comp rx=0 tx=0).stop 2024-04-26T20:18:28.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 msgr2=0x7fe77013e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe76800c730 tx=0x7fe76800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 shutdown_connections 2024-04-26T20:18:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe770142600 0x7fe77013e280 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7700cb6e0 0x7fe7700b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe74c06ca10 0x7fe74c06eed0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 --2- 172.21.15.88:0/3710528715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe77006fda0 0x7fe77013c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.640+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 >> 172.21.15.88:0/3710528715 conn(0x7fe7700d0b50 msgr2=0x7fe7700b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.642+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 shutdown_connections 2024-04-26T20:18:28.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.643+0000 7fe776600640 1 -- 172.21.15.88:0/3710528715 wait complete. 2024-04-26T20:18:28.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:18:28.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.864+0000 7f3e1d400640 1 Processor -- start 2024-04-26T20:18:28.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.865+0000 7f3e1d400640 1 -- start start 2024-04-26T20:18:28.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e1d400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3e180b6170 0x7f3e180c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e1d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d1da0 0x7f3e18162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e1d400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e180d3830 con 0x7f3e180d19c0 2024-04-26T20:18:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e1d400640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3e180d40d0 con 0x7f3e180b6170 2024-04-26T20:18:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e1d400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3e180d3b10 con 0x7f3e180d1da0 2024-04-26T20:18:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e16800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f3e180d19c0 0x7f3e180d2f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59754/0 (socket says 172.21.15.88:59754) 2024-04-26T20:18:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.866+0000 7f3e16800640 1 -- 172.21.15.88:0/2967231386 learned_addr learned my addr 172.21.15.88:0/2967231386 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:28.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e0de00640 1 --2- 172.21.15.88:0/2967231386 >> v2:172.21.15.88:3300/0 conn(0x7f3e180b6170 0x7f3e180c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e17200640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d1da0 0x7f3e18162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e15e00640 1 -- 172.21.15.88:0/2967231386 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844152896 0 0) 0x7f3e180d3830 con 0x7f3e180d19c0 2024-04-26T20:18:28.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e15e00640 1 -- 172.21.15.88:0/2967231386 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df8003520 con 0x7f3e180d19c0 2024-04-26T20:18:28.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e0de00640 1 -- 172.21.15.88:0/2967231386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d1da0 msgr2=0x7f3e18162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e0de00640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d1da0 0x7f3e18162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e0de00640 1 -- 172.21.15.88:0/2967231386 >> v1:172.21.15.88:6789/0 conn(0x7f3e180d19c0 legacy=0x7f3e180d2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.867+0000 7f3e0de00640 1 -- 172.21.15.88:0/2967231386 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e180d3df0 con 0x7f3e180b6170 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.868+0000 7f3e0de00640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180b6170 0x7f3e180c8c70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3e000097a0 tx=0x7f3e0002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3444c1bc4b3192b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e15e00640 1 -- 172.21.15.88:0/2967231386 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00037070 con 0x7f3e180b6170 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e15e00640 1 -- 172.21.15.88:0/2967231386 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0002ae50 con 0x7f3e180b6170 2024-04-26T20:18:28.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e15e00640 1 -- 172.21.15.88:0/2967231386 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00032660 con 0x7f3e180b6170 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 -- 172.21.15.88:0/2967231386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180b6170 msgr2=0x7f3e180c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180b6170 0x7f3e180c8c70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3e000097a0 tx=0x7f3e0002a100 comp rx=0 tx=0).stop 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 -- 172.21.15.88:0/2967231386 shutdown_connections 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d1da0 0x7f3e18162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2967231386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180b6170 0x7f3e180c8c70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 -- 172.21.15.88:0/2967231386 >> 172.21.15.88:0/2967231386 conn(0x7f3e1809e460 msgr2=0x7f3e180d5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:28.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.869+0000 7f3e1d400640 1 -- 172.21.15.88:0/2967231386 shutdown_connections 2024-04-26T20:18:28.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.870+0000 7f3e1d400640 1 -- 172.21.15.88:0/2967231386 wait complete. 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.871+0000 7f3e1d400640 1 Processor -- start 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.871+0000 7f3e1d400640 1 -- start start 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e180b6170 0x7f3e180b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 0x7f3e18154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18162b80 con 0x7f3e180d1da0 2024-04-26T20:18:28.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3e180cfc30 con 0x7f3e180b6170 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.872+0000 7f3e1d400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18162d00 con 0x7f3e180d19c0 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56780/0 (socket says 172.21.15.88:56780) 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 -- 172.21.15.88:0/2435432140 learned_addr learned my addr 172.21.15.88:0/2435432140 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e16800640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e180b6170 0x7f3e180b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e0de00640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 0x7f3e18154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 -- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 msgr2=0x7f3e18154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 0x7f3e18154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 -- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e180b6170 msgr2=0x7f3e180b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e180b6170 0x7f3e180b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:28.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 -- 172.21.15.88:0/2435432140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e180b52e0 con 0x7f3e180d1da0 2024-04-26T20:18:28.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e0de00640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 0x7f3e18154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:28.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.873+0000 7f3e17200640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e10007e60 tx=0x7f3e1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.874+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10014420 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.874+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e10019070 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.874+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10014820 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.874+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e180b3d00 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.874+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e180b42c0 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.875+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e18069650 con 0x7f3e180d1da0 2024-04-26T20:18:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.876+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f3e100149c0 con 0x7f3e180d1da0 2024-04-26T20:18:28.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.877+0000 7f3e0f400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 0x7f3df406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:28.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.877+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f3e1008c380 con 0x7f3e180d1da0 2024-04-26T20:18:28.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.877+0000 7f3e16800640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 0x7f3df406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:28.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.878+0000 7f3e16800640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 0x7f3df406eed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005880 tx=0x7f3e08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:28.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:28.881+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e1005a870 con 0x7f3e180d1da0 2024-04-26T20:18:29.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.081+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3e180d26d0 con 0x7f3e180d1da0 2024-04-26T20:18:29.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.088+0000 7f3e0f400640 1 -- 172.21.15.88:0/2435432140 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f3e1005a170 con 0x7f3e180d1da0 2024-04-26T20:18:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 msgr2=0x7f3df406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 0x7f3df406eed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005880 tx=0x7f3e08009040 comp rx=0 tx=0).stop 2024-04-26T20:18:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 msgr2=0x7f3e1809edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e10007e60 tx=0x7f3e1000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 shutdown_connections 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180d1da0 0x7f3e1809edd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3df406ca10 0x7f3df406eed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3e180d19c0 0x7f3e18154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 --2- 172.21.15.88:0/2435432140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e180b6170 0x7f3e180b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 >> 172.21.15.88:0/2435432140 conn(0x7f3e1809e460 msgr2=0x7f3e180d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:29.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.091+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 shutdown_connections 2024-04-26T20:18:29.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.092+0000 7f3e1d400640 1 -- 172.21.15.88:0/2435432140 wait complete. 2024-04-26T20:18:29.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2024-04-26T20:18:29.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.310+0000 7fb6f5800640 1 Processor -- start 2024-04-26T20:18:29.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.310+0000 7fb6f5800640 1 -- start start 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6f5800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb6f0073720 0x7fb6f006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6f5800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fb6f015dec0 0x7fb6f0073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6f5800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f0074040 con 0x7fb6f006ed40 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6f5800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f00701a0 con 0x7fb6f0073720 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6f5800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f015e320 con 0x7fb6f015dec0 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6efe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb6f0073720 0x7fb6f006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6efe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb6f0073720 0x7fb6f006fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56784/0 (socket says 172.21.15.88:56784) 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6efe00640 1 -- 172.21.15.88:0/2887508430 learned_addr learned my addr 172.21.15.88:0/2887508430 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:29.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.312+0000 7fb6ef400640 1 --2- 172.21.15.88:0/2887508430 >> v2:172.21.15.132:3300/0 conn(0x7fb6f015dec0 0x7fb6f0073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.313+0000 7fb6efe00640 1 -- 172.21.15.88:0/2887508430 >> v2:172.21.15.132:3300/0 conn(0x7fb6f015dec0 msgr2=0x7fb6f0073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:29.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.313+0000 7fb6efe00640 1 --2- 172.21.15.88:0/2887508430 >> v2:172.21.15.132:3300/0 conn(0x7fb6f015dec0 0x7fb6f0073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.313+0000 7fb6efe00640 1 -- 172.21.15.88:0/2887508430 >> v1:172.21.15.132:6789/0 conn(0x7fb6f006ed40 legacy=0x7fb6f00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:29.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.313+0000 7fb6efe00640 1 -- 172.21.15.88:0/2887508430 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f006df80 con 0x7fb6f0073720 2024-04-26T20:18:29.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6efe00640 1 --2- 172.21.15.88:0/2887508430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f006fa90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0075ec0 tx=0x7fb6e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3432894ef316343 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:29.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6eea00640 1 -- 172.21.15.88:0/2887508430 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0037070 con 0x7fb6f0073720 2024-04-26T20:18:29.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6eea00640 1 -- 172.21.15.88:0/2887508430 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0032440 con 0x7fb6f0073720 2024-04-26T20:18:29.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6eea00640 1 -- 172.21.15.88:0/2887508430 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0032810 con 0x7fb6f0073720 2024-04-26T20:18:29.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6f5800640 1 -- 172.21.15.88:0/2887508430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 msgr2=0x7fb6f006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.314+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2887508430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f006fa90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0075ec0 tx=0x7fb6e002a230 comp rx=0 tx=0).stop 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 -- 172.21.15.88:0/2887508430 shutdown_connections 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2887508430 >> v2:172.21.15.132:3300/0 conn(0x7fb6f015dec0 0x7fb6f0073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2887508430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f006fa90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 -- 172.21.15.88:0/2887508430 >> 172.21.15.88:0/2887508430 conn(0x7fb6f00d0ae0 msgr2=0x7fb6f00d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 -- 172.21.15.88:0/2887508430 shutdown_connections 2024-04-26T20:18:29.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.315+0000 7fb6f5800640 1 -- 172.21.15.88:0/2887508430 wait complete. 2024-04-26T20:18:29.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.316+0000 7fb6f5800640 1 Processor -- start 2024-04-26T20:18:29.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.317+0000 7fb6f5800640 1 -- start start 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.317+0000 7fb6f5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f006ed40 0x7fb6f00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.317+0000 7fb6f5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.317+0000 7fb6f5800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 0x7fb6f00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6f5800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f006f1f0 con 0x7fb6f0073720 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6f5800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f006e540 con 0x7fb6f015dec0 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6f5800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f0146910 con 0x7fb6f006ed40 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56796/0 (socket says 172.21.15.88:56796) 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 -- 172.21.15.88:0/2273855731 learned_addr learned my addr 172.21.15.88:0/2273855731 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6efe00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f006ed40 0x7fb6f00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6f4e00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 0x7fb6f00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 -- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f006ed40 msgr2=0x7fb6f00b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f006ed40 0x7fb6f00b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 -- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 msgr2=0x7fb6f00ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 0x7fb6f00ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6ef400640 1 -- 172.21.15.88:0/2273855731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00cf0c0 con 0x7fb6f0073720 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.318+0000 7fb6f4e00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 0x7fb6f00ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6ef400640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00d7d0 tx=0x7fb6dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc004020 con 0x7fb6f0073720 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6dc0041e0 con 0x7fb6f0073720 2024-04-26T20:18:29.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f00cf3a0 con 0x7fb6f0073720 2024-04-26T20:18:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0043d0 con 0x7fb6f0073720 2024-04-26T20:18:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.319+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f00b5190 con 0x7fb6f0073720 2024-04-26T20:18:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.320+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f00cf8c0 con 0x7fb6f0073720 2024-04-26T20:18:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.321+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fb6dc01b020 con 0x7fb6f0073720 2024-04-26T20:18:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.322+0000 7fb6e7e00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 0x7fb6d006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.322+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fb6dc08b840 con 0x7fb6f0073720 2024-04-26T20:18:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.323+0000 7fb6efe00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 0x7fb6d006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.323+0000 7fb6efe00640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 0x7fb6d006eed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0034000 tx=0x7fb6e002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:29.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.327+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb6dc059e60 con 0x7fb6f0073720 2024-04-26T20:18:29.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.598+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) -- 0x7fb6f00b1a00 con 0x7fb6f0073720 2024-04-26T20:18:29.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.653+0000 7fb6e7e00640 1 -- 172.21.15.88:0/2273855731 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v680) ==== 158+0+0 (secure 0 0 0) 0x7fb6dc059760 con 0x7fb6f0073720 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 msgr2=0x7fb6d006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 0x7fb6d006eed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0034000 tx=0x7fb6e002e000 comp rx=0 tx=0).stop 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 msgr2=0x7fb6f00ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00d7d0 tx=0x7fb6dc00dc90 comp rx=0 tx=0).stop 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 shutdown_connections 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6f015dec0 0x7fb6f00ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f0073720 0x7fb6f00ca0a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb6d006ca10 0x7fb6d006eed0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 --2- 172.21.15.88:0/2273855731 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb6f006ed40 0x7fb6f00b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 >> 172.21.15.88:0/2273855731 conn(0x7fb6f00d0ae0 msgr2=0x7fb6f00b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 shutdown_connections 2024-04-26T20:18:29.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:29.656+0000 7fb6f5800640 1 -- 172.21.15.88:0/2273855731 wait complete. 2024-04-26T20:18:29.680 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ERROR 2024-04-26T20:18:29.680 INFO:tasks.cephfs_test_runner: 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner:ERROR: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 572, in test_volume_info_pending_subvol_deletions 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "rm", self.volname, subvolname) 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T20:18:29.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi088 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol92860' 2024-04-26T20:18:29.682 INFO:tasks.cephfs_test_runner: 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:Ran 3 tests in 1083.223s 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:ERROR: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 572, in test_volume_info_pending_subvol_deletions 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: self._fs_cmd("subvolume", "rm", self.volname, subvolname) 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs/test_volumes.py", line 33, in _fs_cmd 2024-04-26T20:18:29.683 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue().strip() 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi088 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol92860' 2024-04-26T20:18:29.684 INFO:tasks.cephfs_test_runner: 2024-04-26T20:18:29.685 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:18:29.957 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fddcc58813184e8aaa0b94c42a0d0a2f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2024-04-26T20:18:29.958 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T20:18:29.966 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T20:18:30.009 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T20:18:30.010 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:18:30.010 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:30.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:18:30.069 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-26T20:18:30.069 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:30.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052732,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:18:30.120 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:18:30.120 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:30.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:18:30.180 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-04-26T20:18:30.180 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:18:30.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052736,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498008k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:18:30.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.233 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-04-26T20:18:30.287 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:18:30.287 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:18:30.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:18:30.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.374 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:18:30.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.448 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-04-26T20:18:30.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.553 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-04-26T20:18:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:18:30.612 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:18:30.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.613 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:18:30.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.700 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:18:30.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:18:30.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-04-26T20:18:30.903 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T20:18:30.912 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:18:30.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:18:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.086+0000 7f291e600640 1 Processor -- start 2024-04-26T20:18:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.087+0000 7f291e600640 1 -- start start 2024-04-26T20:18:31.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.088+0000 7f291e600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f291815cdf0 0x7f29180add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.088+0000 7f291e600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f29180cd650 0x7f291806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.088+0000 7f291e600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29180af9a0 con 0x7f29180b4f50 2024-04-26T20:18:31.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.088+0000 7f291e600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f29180ae2c0 con 0x7f29180cd650 2024-04-26T20:18:31.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.088+0000 7f291e600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f29180b05b0 con 0x7f291815cdf0 2024-04-26T20:18:31.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f290ea00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f29180cd650 0x7f291806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f2917e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f29180b4f50 0x7f2918072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59756/0 (socket says 172.21.15.88:59756) 2024-04-26T20:18:31.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f2917e00640 1 -- 172.21.15.88:0/742317669 learned_addr learned my addr 172.21.15.88:0/742317669 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:31.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f2917400640 1 --2- 172.21.15.88:0/742317669 >> v2:172.21.15.132:3300/0 conn(0x7f291815cdf0 0x7f29180add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f290ea00640 1 -- 172.21.15.88:0/742317669 >> v2:172.21.15.132:3300/0 conn(0x7f291815cdf0 msgr2=0x7f29180add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f290ea00640 1 --2- 172.21.15.88:0/742317669 >> v2:172.21.15.132:3300/0 conn(0x7f291815cdf0 0x7f29180add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f2916a00640 1 -- 172.21.15.88:0/742317669 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3349143068 0 0) 0x7f29180af9a0 con 0x7f29180b4f50 2024-04-26T20:18:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f290ea00640 1 -- 172.21.15.88:0/742317669 >> v1:172.21.15.88:6789/0 conn(0x7f29180b4f50 legacy=0x7f2918072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.089+0000 7f290ea00640 1 -- 172.21.15.88:0/742317669 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180afc80 con 0x7f29180cd650 2024-04-26T20:18:31.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.090+0000 7f290ea00640 1 --2- 172.21.15.88:0/742317669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180cd650 0x7f291806c770 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f29180b3950 tx=0x7f290002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c719a747ae1faa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f2916a00640 1 -- 172.21.15.88:0/742317669 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900037070 con 0x7f29180cd650 2024-04-26T20:18:31.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f2916a00640 1 -- 172.21.15.88:0/742317669 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2900032440 con 0x7f29180cd650 2024-04-26T20:18:31.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f2916a00640 1 -- 172.21.15.88:0/742317669 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900032810 con 0x7f29180cd650 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 -- 172.21.15.88:0/742317669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180cd650 msgr2=0x7f291806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 --2- 172.21.15.88:0/742317669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180cd650 0x7f291806c770 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f29180b3950 tx=0x7f290002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 -- 172.21.15.88:0/742317669 shutdown_connections 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 --2- 172.21.15.88:0/742317669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180cd650 0x7f291806c770 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 --2- 172.21.15.88:0/742317669 >> v2:172.21.15.132:3300/0 conn(0x7f291815cdf0 0x7f29180add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 -- 172.21.15.88:0/742317669 >> 172.21.15.88:0/742317669 conn(0x7f29180b2d50 msgr2=0x7f29180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 -- 172.21.15.88:0/742317669 shutdown_connections 2024-04-26T20:18:31.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.091+0000 7f291e600640 1 -- 172.21.15.88:0/742317669 wait complete. 2024-04-26T20:18:31.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.093+0000 7f291e600640 1 Processor -- start 2024-04-26T20:18:31.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.093+0000 7f291e600640 1 -- start start 2024-04-26T20:18:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 0x7f29181515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291815cdf0 0x7f29180ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f291815d1d0 con 0x7f29180b4f50 2024-04-26T20:18:31.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29180b0240 con 0x7f291815cdf0 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f291e600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f291815d350 con 0x7f29180cd650 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f2917400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.094+0000 7f2917400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56816/0 (socket says 172.21.15.88:56816) 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 -- 172.21.15.88:0/1455530956 learned_addr learned my addr 172.21.15.88:0/1455530956 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f290ea00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 0x7f29181515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917e00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291815cdf0 0x7f29180ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 -- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 msgr2=0x7f29181515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 0x7f29181515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 -- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291815cdf0 msgr2=0x7f29180ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291815cdf0 0x7f29180ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 -- 172.21.15.88:0/1455530956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291815e270 con 0x7f29180b4f50 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f290ea00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 0x7f29181515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f2917400640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f290800d5f0 tx=0x7f290800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.095+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080125f0 con 0x7f29180b4f50 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.096+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2908019070 con 0x7f29180b4f50 2024-04-26T20:18:31.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.096+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291815d9d0 con 0x7f29180b4f50 2024-04-26T20:18:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.096+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080129d0 con 0x7f29180b4f50 2024-04-26T20:18:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.096+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291815df90 con 0x7f29180b4f50 2024-04-26T20:18:31.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.097+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2918069540 con 0x7f29180b4f50 2024-04-26T20:18:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.098+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f2908012b70 con 0x7f29180b4f50 2024-04-26T20:18:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.099+0000 7f290fe00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 0x7f28f406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.099+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f290808be80 con 0x7f29180b4f50 2024-04-26T20:18:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.099+0000 7f290ea00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 0x7f28f406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.100+0000 7f290ea00640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 0x7f28f406eed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2900034000 tx=0x7f290002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.103+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f290805a400 con 0x7f29180b4f50 2024-04-26T20:18:31.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.300+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f291809cd00 con 0x7f28f406ca10 2024-04-26T20:18:31.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.302+0000 7f290fe00640 1 -- 172.21.15.88:0/1455530956 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13409 (secure 0 0 0) 0x7f291809cd00 con 0x7f28f406ca10 2024-04-26T20:18:31.303 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:31.303 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:18:31.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 msgr2=0x7f28f406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 0x7f28f406eed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2900034000 tx=0x7f290002e000 comp rx=0 tx=0).stop 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 msgr2=0x7f29180cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f290800d5f0 tx=0x7f290800dab0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 shutdown_connections 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f291815cdf0 0x7f29180ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f28f406ca10 0x7f28f406eed0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29180cd650 0x7f29181515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 --2- 172.21.15.88:0/1455530956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29180b4f50 0x7f29180cfb50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 >> 172.21.15.88:0/1455530956 conn(0x7f29180b2d50 msgr2=0x7f29180d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 shutdown_connections 2024-04-26T20:18:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.305+0000 7f291e600640 1 -- 172.21.15.88:0/1455530956 wait complete. 2024-04-26T20:18:31.323 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":628,"stamp":"2024-04-26T20:18:29.337752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290288,"kb_used_data":36720,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754684432,"statfs":{"total":773094113280,"available":772796858368,"internally_reserved":0,"allocated":37601280,"data_stored":33928486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15013395,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3744,"num_write_kb":-15986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001940"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":263,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-26T20:15:40.155528+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:15:40.155528+0000","last_peered":"2024-04-26T20:15:40.155528+0000","last_clean":"2024-04-26T20:15:40.155528+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:15:40.155528+0000","last_undegraded":"2024-04-26T20:15:40.155528+0000","last_fullsized":"2024-04-26T20:15:40.155528+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34912,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336928,"statfs":{"total":96636764160,"available":96601014272,"internally_reserved":0,"allocated":3825664,"data_stored":3360117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":4876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":4993024,"data_stored":4534516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":798720,"data_stored":340340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575076,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":4872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":4988928,"data_stored":4534494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":4876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":4993024,"data_stored":4534536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":4876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":4993024,"data_stored":4535096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575076,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46688,"kb_used_data":11352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94325152,"statfs":{"total":96636764160,"available":96588955648,"internally_reserved":0,"allocated":11624448,"data_stored":11158899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1384448,"data_stored":930488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:18:31.324 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:18:31.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.521+0000 7fa6e1400640 1 Processor -- start 2024-04-26T20:18:31.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.522+0000 7fa6e1400640 1 -- start start 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6e1400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa6dc06f230 0x7fa6dc0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6e1400640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc070670 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6e1400640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6dc070950 con 0x7fa6dc09ba50 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6e1400640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa6dc070420 con 0x7fa6dc06f230 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6d1e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa6dc0b58a0 0x7fa6dc0d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59760/0 (socket says 172.21.15.88:59760) 2024-04-26T20:18:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.523+0000 7fa6d1e00640 1 -- 172.21.15.88:0/4238748868 learned_addr learned my addr 172.21.15.88:0/4238748868 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6db200640 1 --2- 172.21.15.88:0/4238748868 >> v2:172.21.15.132:3300/0 conn(0x7fa6dc06f230 0x7fa6dc0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242182209 0 0) 0x7fa6dc070670 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6c0003520 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2092448176 0 0) 0x7fa6c0003520 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 >> v2:172.21.15.132:3300/0 conn(0x7fa6dc06f230 msgr2=0x7fa6dc0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 --2- 172.21.15.88:0/4238748868 >> v2:172.21.15.132:3300/0 conn(0x7fa6dc06f230 0x7fa6dc0d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 >> v1:172.21.15.132:6789/0 conn(0x7fa6dc09ba50 legacy=0x7fa6dc09be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc06e9e0 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa6c4001670 con 0x7fa6dc0b58a0 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 >> v1:172.21.15.88:6789/0 conn(0x7fa6dc0b58a0 legacy=0x7fa6dc0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.524+0000 7fa6d9e00640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 0x7fa6c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6d9e00640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c0004800 0x7fa6c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6d9e00640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00051c0 0x7fa6c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc070420 con 0x7fa6c00039b0 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c000c160 con 0x7fa6c0004800 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c000c360 con 0x7fa6c00051c0 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 0x7fa6c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 -- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00051c0 msgr2=0x7fa6c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00051c0 0x7fa6c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 -- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c0004800 msgr2=0x7fa6c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6c0004800 0x7fa6c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6da800640 1 -- 172.21.15.88:0/4238748868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c0003520 con 0x7fa6c00039b0 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.525+0000 7fa6db200640 1 -- 172.21.15.88:0/4238748868 reap_dead start 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.526+0000 7fa6da800640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 0x7fa6c0003dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc001750 tx=0x7fa6c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3b1376820c1519 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc001370 con 0x7fa6c00039b0 2024-04-26T20:18:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0052d0 con 0x7fa6c00039b0 2024-04-26T20:18:31.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6d9e00640 1 -- 172.21.15.88:0/4238748868 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc02abc0 con 0x7fa6c00039b0 2024-04-26T20:18:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 -- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 msgr2=0x7fa6c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 0x7fa6c0003dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc001750 tx=0x7fa6c0005830 comp rx=0 tx=0).stop 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 -- 172.21.15.88:0/4238748868 shutdown_connections 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 --2- 172.21.15.88:0/4238748868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6c00039b0 0x7fa6c0003dd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 -- 172.21.15.88:0/4238748868 >> 172.21.15.88:0/4238748868 conn(0x7fa6dc0b4b70 msgr2=0x7fa6dc0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 -- 172.21.15.88:0/4238748868 shutdown_connections 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.527+0000 7fa6e1400640 1 -- 172.21.15.88:0/4238748868 wait complete. 2024-04-26T20:18:31.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.529+0000 7fa6e1400640 1 Processor -- start 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.529+0000 7fa6e1400640 1 -- start start 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 0x7fa6dc1e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6dc0c81e0 0x7fa6dc0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc0cf440 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa6dc06e790 con 0x7fa6dc0c81e0 2024-04-26T20:18:31.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6e1400640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa6dc0d2d00 con 0x7fa6c00039b0 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.530+0000 7fa6d1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56834/0 (socket says 172.21.15.88:56834) 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 -- 172.21.15.88:0/2333350687 learned_addr learned my addr 172.21.15.88:0/2333350687 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6da800640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 0x7fa6dc1e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 -- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 msgr2=0x7fa6dc1e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 0x7fa6dc1e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 -- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6dc0c81e0 msgr2=0x7fa6dc0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6dc0c81e0 0x7fa6dc0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 -- 172.21.15.88:0/2333350687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6dc1e4ea0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6da800640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 0x7fa6dc1e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.531+0000 7fa6d1e00640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c770 tx=0x7fa6c400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.532+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4018070 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.532+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c40074e0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.532+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4004a20 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.532+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6dc0cd210 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.532+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6dc0cd7d0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.533+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6dc069540 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.534+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fa6c4004bc0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.535+0000 7fa6d3400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 0x7fa6b806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.535+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fa6c408aef0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.535+0000 7fa6da800640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 0x7fa6b806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.536+0000 7fa6da800640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 0x7fa6b806ef00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc032000 tx=0x7fa6cc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.539+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6c40593e0 con 0x7fa6dc0ccd50 2024-04-26T20:18:31.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.734+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6dc0b4950 con 0x7fa6b806ca40 2024-04-26T20:18:31.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.736+0000 7fa6d3400640 1 -- 172.21.15.88:0/2333350687 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fa6dc0b4950 con 0x7fa6b806ca40 2024-04-26T20:18:31.737 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:31.737 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:18:31.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.738+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 msgr2=0x7fa6b806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.738+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 0x7fa6b806ef00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc032000 tx=0x7fa6cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.738+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 msgr2=0x7fa6dc0b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.738+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c770 tx=0x7fa6c400cc30 comp rx=0 tx=0).stop 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 shutdown_connections 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6dc0c81e0 0x7fa6dc0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6dc0ccd50 0x7fa6dc0b68e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa6b806ca40 0x7fa6b806ef00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 --2- 172.21.15.88:0/2333350687 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa6c00039b0 0x7fa6dc1e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 >> 172.21.15.88:0/2333350687 conn(0x7fa6dc0b4b70 msgr2=0x7fa6dc0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 shutdown_connections 2024-04-26T20:18:31.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.739+0000 7fa6e1400640 1 -- 172.21.15.88:0/2333350687 wait complete. 2024-04-26T20:18:31.758 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2024-04-26T20:18:31.338055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261244,"kb_used_data":7628,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713476,"statfs":{"total":773094113280,"available":772826599424,"internally_reserved":0,"allocated":7811072,"data_stored":4021822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15013395,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3744,"num_write_kb":-15986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.137793"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":269,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T20:18:29.348236+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:18:29.348236+0000","last_peered":"2024-04-26T20:18:29.348236+0000","last_clean":"2024-04-26T20:18:29.348236+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:18:29.348236+0000","last_undegraded":"2024-04-26T20:18:29.348236+0000","last_fullsized":"2024-04-26T20:18:29.348236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":352450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575078,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575079,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":352320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36140,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":819200,"data_stored":352320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":942688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:18:31.759 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:18:31.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:18:31.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.954+0000 7efc76a00640 1 Processor -- start 2024-04-26T20:18:31.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.954+0000 7efc76a00640 1 -- start start 2024-04-26T20:18:31.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc76a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc700d5d90 0x7efc7015f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc76a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc700b5d30 con 0x7efc700d34a0 2024-04-26T20:18:31.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc76a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc700b6010 con 0x7efc7015ef60 2024-04-26T20:18:31.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc76a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7efc700b62f0 con 0x7efc700d5d90 2024-04-26T20:18:31.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc74a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7efc700d34a0 0x7efc700d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59774/0 (socket says 172.21.15.88:59774) 2024-04-26T20:18:31.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc67e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7efc700d5d90 0x7efc7015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.956+0000 7efc74a00640 1 -- 172.21.15.88:0/3569276820 learned_addr learned my addr 172.21.15.88:0/3569276820 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:31.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.957+0000 7efc67e00640 1 -- 172.21.15.88:0/3569276820 >> v1:172.21.15.132:6789/0 conn(0x7efc7015ef60 legacy=0x7efc700b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:31.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.957+0000 7efc67e00640 1 -- 172.21.15.88:0/3569276820 >> v1:172.21.15.88:6789/0 conn(0x7efc700d34a0 legacy=0x7efc700d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.957+0000 7efc67e00640 1 -- 172.21.15.88:0/3569276820 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc700b65d0 con 0x7efc700d5d90 2024-04-26T20:18:31.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.957+0000 7efc6f400640 1 -- 172.21.15.88:0/3569276820 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952462780 0 0) 0x7efc700b5d30 con 0x7efc700d34a0 2024-04-26T20:18:31.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.958+0000 7efc67e00640 1 --2- 172.21.15.88:0/3569276820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d5d90 0x7efc7015f340 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc580097a0 tx=0x7efc5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea086179f068303 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.958+0000 7efc6f400640 1 -- 172.21.15.88:0/3569276820 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58037070 con 0x7efc700d5d90 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.958+0000 7efc6f400640 1 -- 172.21.15.88:0/3569276820 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc58032440 con 0x7efc700d5d90 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.958+0000 7efc6f400640 1 -- 172.21.15.88:0/3569276820 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58032810 con 0x7efc700d5d90 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 -- 172.21.15.88:0/3569276820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d5d90 msgr2=0x7efc7015f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 --2- 172.21.15.88:0/3569276820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d5d90 0x7efc7015f340 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc580097a0 tx=0x7efc5802a270 comp rx=0 tx=0).stop 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 -- 172.21.15.88:0/3569276820 shutdown_connections 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 --2- 172.21.15.88:0/3569276820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d5d90 0x7efc7015f340 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 -- 172.21.15.88:0/3569276820 >> 172.21.15.88:0/3569276820 conn(0x7efc7009e3b0 msgr2=0x7efc700d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 -- 172.21.15.88:0/3569276820 shutdown_connections 2024-04-26T20:18:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.959+0000 7efc76a00640 1 -- 172.21.15.88:0/3569276820 wait complete. 2024-04-26T20:18:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.960+0000 7efc76a00640 1 Processor -- start 2024-04-26T20:18:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.961+0000 7efc76a00640 1 -- start start 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc700d5d90 0x7efc70152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc7015ef60 0x7efc700ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc700d3ab0 con 0x7efc700d34a0 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7efc700d3930 con 0x7efc700d5d90 2024-04-26T20:18:31.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc76a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7efc700d3c30 con 0x7efc7015ef60 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56864/0 (socket says 172.21.15.88:56864) 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 -- 172.21.15.88:0/1051576491 learned_addr learned my addr 172.21.15.88:0/1051576491 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc67e00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc700d5d90 0x7efc70152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 -- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc7015ef60 msgr2=0x7efc700ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc7015ef60 0x7efc700ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 -- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc700d5d90 msgr2=0x7efc70152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:31.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc700d5d90 0x7efc70152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.962+0000 7efc6fe00640 1 -- 172.21.15.88:0/1051576491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc701f9690 con 0x7efc700d34a0 2024-04-26T20:18:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc6fe00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c430 tx=0x7efc6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60011790 con 0x7efc700d34a0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc60017070 con 0x7efc700d34a0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60011b70 con 0x7efc700d34a0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc70150010 con 0x7efc700d34a0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.963+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701505d0 con 0x7efc700d34a0 2024-04-26T20:18:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.964+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc70069540 con 0x7efc700d34a0 2024-04-26T20:18:31.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.965+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7efc60011d10 con 0x7efc700d34a0 2024-04-26T20:18:31.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.966+0000 7efc6cc00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 0x7efc4c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:31.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.966+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7efc6008b930 con 0x7efc700d34a0 2024-04-26T20:18:31.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.966+0000 7efc67e00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 0x7efc4c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:31.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.967+0000 7efc67e00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 0x7efc4c06eed0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc58034000 tx=0x7efc5802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:31.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:31.970+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc701f9690 con 0x7efc700d34a0 2024-04-26T20:18:32.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.164+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc7008d090 con 0x7efc4c06ca10 2024-04-26T20:18:32.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.166+0000 7efc6cc00640 1 -- 172.21.15.88:0/1051576491 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7efc7008d090 con 0x7efc4c06ca10 2024-04-26T20:18:32.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:32.167 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 msgr2=0x7efc4c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 0x7efc4c06eed0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efc58034000 tx=0x7efc5802e000 comp rx=0 tx=0).stop 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 msgr2=0x7efc701f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc6000c430 tx=0x7efc6000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 shutdown_connections 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7efc4c06ca10 0x7efc4c06eed0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7efc7015ef60 0x7efc700ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc700d5d90 0x7efc70152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 --2- 172.21.15.88:0/1051576491 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc700d34a0 0x7efc701f9150 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 >> 172.21.15.88:0/1051576491 conn(0x7efc7009e3b0 msgr2=0x7efc700b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:32.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 shutdown_connections 2024-04-26T20:18:32.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.168+0000 7efc76a00640 1 -- 172.21.15.88:0/1051576491 wait complete. 2024-04-26T20:18:32.187 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":629,"stamp":"2024-04-26T20:18:31.338055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261244,"kb_used_data":7628,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713476,"statfs":{"total":773094113280,"available":772826599424,"internally_reserved":0,"allocated":7811072,"data_stored":4021822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15013395,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-3744,"num_write_kb":-15986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.137793"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":269,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T20:18:29.348236+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:18:29.348236+0000","last_peered":"2024-04-26T20:18:29.348236+0000","last_clean":"2024-04-26T20:18:29.348236+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:18:29.348236+0000","last_undegraded":"2024-04-26T20:18:29.348236+0000","last_fullsized":"2024-04-26T20:18:29.348236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":352450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575078,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575079,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":352320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36140,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":819200,"data_stored":352320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1413120,"data_stored":942688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:18:32.188 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:18:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.380+0000 7f29a5600640 1 Processor -- start 2024-04-26T20:18:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.380+0000 7f29a5600640 1 -- start start 2024-04-26T20:18:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f29a5600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f29a006c770 0x7f29a006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f29a5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f29a0073120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f29a5600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29a006d090 con 0x7f29a015cfd0 2024-04-26T20:18:32.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f29a5600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f29a00af460 con 0x7f29a006c770 2024-04-26T20:18:32.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f29a5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f29a00ae850 con 0x7f29a015cb70 2024-04-26T20:18:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f299f400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f29a015cfd0 0x7f29a0073660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59782/0 (socket says 172.21.15.88:59782) 2024-04-26T20:18:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.383+0000 7f299f400640 1 -- 172.21.15.88:0/1812196601 learned_addr learned my addr 172.21.15.88:0/1812196601 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344994036 0 0) 0x7f29a006d090 con 0x7f29a015cfd0 2024-04-26T20:18:32.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f2997e00640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f29a0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299ea00640 1 --2- 172.21.15.88:0/1812196601 >> v2:172.21.15.132:3300/0 conn(0x7f29a006c770 0x7f29a006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2984005730 con 0x7f29a015cfd0 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4003167547 0 0) 0x7f2984005730 con 0x7f29a015cfd0 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 msgr2=0x7f29a0073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f29a0073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 >> v2:172.21.15.132:3300/0 conn(0x7f29a006c770 msgr2=0x7f29a006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:32.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 --2- 172.21.15.88:0/1812196601 >> v2:172.21.15.132:3300/0 conn(0x7f29a006c770 0x7f29a006cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a00aeb30 con 0x7f29a015cfd0 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f29980019f0 con 0x7f29a015cfd0 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 >> v1:172.21.15.88:6789/0 conn(0x7f29a015cfd0 legacy=0x7f29a0073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f2984005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2984006140 0x7f2984006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2984006f50 0x7f29840073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29a00af460 con 0x7f2984006f50 2024-04-26T20:18:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29a00ae850 con 0x7f2984006140 2024-04-26T20:18:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.384+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f298400dbd0 con 0x7f29a015cb70 2024-04-26T20:18:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2984006f50 0x7f29840073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299ea00640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f2984005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 -- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 msgr2=0x7f2984005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f29a015cb70 0x7f2984005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 -- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2984006140 msgr2=0x7f2984006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f2997e00640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2984006140 0x7f2984006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2984006140 0x7f2984006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.385+0000 7f299f400640 1 -- 172.21.15.88:0/1812196601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2984005730 con 0x7f2984006f50 2024-04-26T20:18:32.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.386+0000 7f299f400640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2984006f50 0x7f29840073d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f299800a8a0 tx=0x7f2984007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bddffa147fbf7d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.386+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800a4c0 con 0x7f2984006f50 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299802f070 con 0x7f2984006f50 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f299e000640 1 -- 172.21.15.88:0/1812196601 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299802bd50 con 0x7f2984006f50 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 -- 172.21.15.88:0/1812196601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2984006f50 msgr2=0x7f29840073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 --2- 172.21.15.88:0/1812196601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2984006f50 0x7f29840073d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f299800a8a0 tx=0x7f2984007a40 comp rx=0 tx=0).stop 2024-04-26T20:18:32.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f299f400640 1 -- 172.21.15.88:0/1812196601 reap_dead start 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 -- 172.21.15.88:0/1812196601 shutdown_connections 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 -- 172.21.15.88:0/1812196601 >> 172.21.15.88:0/1812196601 conn(0x7f29a00b2d50 msgr2=0x7f29a00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 -- 172.21.15.88:0/1812196601 shutdown_connections 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.387+0000 7f29a5600640 1 -- 172.21.15.88:0/1812196601 wait complete. 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.389+0000 7f29a5600640 1 Processor -- start 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.389+0000 7f29a5600640 1 -- start start 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2984006f50 0x7f29a014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 0x7f29a006f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29a015c600 con 0x7f29a01e5a80 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f29a00af0f0 con 0x7f29a006f400 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f29a5600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f29a015c780 con 0x7f2984006f50 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f299f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f299f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56886/0 (socket says 172.21.15.88:56886) 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.390+0000 7f299f400640 1 -- 172.21.15.88:0/2770494048 learned_addr learned my addr 172.21.15.88:0/2770494048 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:32.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f2997e00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 0x7f29a006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 -- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2984006f50 msgr2=0x7f29a014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299ea00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2984006f50 0x7f29a014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2984006f50 0x7f29a014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 -- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 msgr2=0x7f29a006f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 0x7f29a006f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 -- 172.21.15.88:0/2770494048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a01e67f0 con 0x7f29a01e5a80 2024-04-26T20:18:32.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f2997e00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 0x7f29a006f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f299f400640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f299802b560 tx=0x7f2998001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800a4c0 con 0x7f29a01e5a80 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f299802bcc0 con 0x7f29a01e5a80 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.391+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998002010 con 0x7f29a01e5a80 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.392+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a01e6ad0 con 0x7f29a01e5a80 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.392+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01e7090 con 0x7f29a01e5a80 2024-04-26T20:18:32.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.393+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a006fdc0 con 0x7f29a01e5a80 2024-04-26T20:18:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.394+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f2998034670 con 0x7f29a01e5a80 2024-04-26T20:18:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.395+0000 7f2996a00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 0x7f297c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:32.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.395+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f29980abc30 con 0x7f29a01e5a80 2024-04-26T20:18:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.396+0000 7f299ea00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 0x7f297c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.396+0000 7f299ea00640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 0x7f297c06eed0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f29900099d0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:32.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.400+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f299807a250 con 0x7f29a01e5a80 2024-04-26T20:18:32.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.605+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29a0068700 con 0x7f29a01e5a80 2024-04-26T20:18:32.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.607+0000 7f2996a00640 1 -- 172.21.15.88:0/2770494048 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+13011 (secure 0 0 0) 0x7f2998079b50 con 0x7f29a01e5a80 2024-04-26T20:18:32.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:32.608 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":68,"fsid":"100f2996-174e-4373-bd56-473fe683ba7a","created":"2024-04-26T19:59:21.824809+0000","modified":"2024-04-26T20:18:28.632519+0000","last_up_change":"2024-04-26T19:59:34.124961+0000","last_in_change":"2024-04-26T19:59:25.173989+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:59:34.974341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd3ad240-72d1-436b-b2ff-4a0233e75d9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6809","nonce":2647855198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6811","nonce":2647855198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6815","nonce":2647855198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2647855198},{"type":"v1","addr":"172.21.15.88:6813","nonce":2647855198}]},"public_addr":"172.21.15.88:6809/2647855198","cluster_addr":"172.21.15.88:6811/2647855198","heartbeat_back_addr":"172.21.15.88:6815/2647855198","heartbeat_front_addr":"172.21.15.88:6813/2647855198","state":["exists","up"]},{"osd":1,"uuid":"5a25ad95-9feb-4a79-be65-5667b5252117","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6801","nonce":125701657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6803","nonce":125701657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6807","nonce":125701657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":125701657},{"type":"v1","addr":"172.21.15.88:6805","nonce":125701657}]},"public_addr":"172.21.15.88:6801/125701657","cluster_addr":"172.21.15.88:6803/125701657","heartbeat_back_addr":"172.21.15.88:6807/125701657","heartbeat_front_addr":"172.21.15.88:6805/125701657","state":["exists","up"]},{"osd":2,"uuid":"cfe0e91a-0fc4-49f8-9cc6-109591be2d7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6821","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6823","nonce":3051227728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6825","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6827","nonce":3051227728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6831","nonce":3051227728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3051227728},{"type":"v1","addr":"172.21.15.88:6829","nonce":3051227728}]},"public_addr":"172.21.15.88:6823/3051227728","cluster_addr":"172.21.15.88:6827/3051227728","heartbeat_back_addr":"172.21.15.88:6831/3051227728","heartbeat_front_addr":"172.21.15.88:6829/3051227728","state":["exists","up"]},{"osd":3,"uuid":"64bb7329-4c70-4470-9975-7573769e432d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6817","nonce":1302198393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6819","nonce":1302198393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6826","nonce":1302198393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":1302198393},{"type":"v1","addr":"172.21.15.88:6822","nonce":1302198393}]},"public_addr":"172.21.15.88:6817/1302198393","cluster_addr":"172.21.15.88:6819/1302198393","heartbeat_back_addr":"172.21.15.88:6826/1302198393","heartbeat_front_addr":"172.21.15.88:6822/1302198393","state":["exists","up"]},{"osd":4,"uuid":"0e15b131-0969-4051-9951-b5b960209504","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6818","nonce":861564477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6822","nonce":861564477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6827","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6829","nonce":861564477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6823","nonce":861564477},{"type":"v1","addr":"172.21.15.132:6825","nonce":861564477}]},"public_addr":"172.21.15.132:6818/861564477","cluster_addr":"172.21.15.132:6822/861564477","heartbeat_back_addr":"172.21.15.132:6829/861564477","heartbeat_front_addr":"172.21.15.132:6825/861564477","state":["exists","up"]},{"osd":5,"uuid":"d47b401d-b4b2-48dc-a68e-3c8ed8ad9b66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6817","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6819","nonce":4023647186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6821","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6824","nonce":4023647186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6831","nonce":4023647186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":4023647186},{"type":"v1","addr":"172.21.15.132:6828","nonce":4023647186}]},"public_addr":"172.21.15.132:6819/4023647186","cluster_addr":"172.21.15.132:6824/4023647186","heartbeat_back_addr":"172.21.15.132:6831/4023647186","heartbeat_front_addr":"172.21.15.132:6828/4023647186","state":["exists","up"]},{"osd":6,"uuid":"c1d094c0-5169-44b9-b13e-2d2f6ece7cb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6801","nonce":2163437982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6803","nonce":2163437982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6807","nonce":2163437982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2163437982},{"type":"v1","addr":"172.21.15.132:6805","nonce":2163437982}]},"public_addr":"172.21.15.132:6801/2163437982","cluster_addr":"172.21.15.132:6803/2163437982","heartbeat_back_addr":"172.21.15.132:6807/2163437982","heartbeat_front_addr":"172.21.15.132:6805/2163437982","state":["exists","up"]},{"osd":7,"uuid":"44bc038e-9483-482f-ac71-acf9b8b70d85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6809","nonce":3391944263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6811","nonce":3391944263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6815","nonce":3391944263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3391944263},{"type":"v1","addr":"172.21.15.132:6813","nonce":3391944263}]},"public_addr":"172.21.15.132:6809/3391944263","cluster_addr":"172.21.15.132:6811/3391944263","heartbeat_back_addr":"172.21.15.132:6815/3391944263","heartbeat_front_addr":"172.21.15.132:6813/3391944263","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.527703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.737069+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.535879+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.591439+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.380354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.666705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.528351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:59:30.583328+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.132:6835/1355496095":"2024-04-27T20:18:25.196269+0000","172.21.15.132:6833/1355496095":"2024-04-27T20:18:25.196269+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 msgr2=0x7f297c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 0x7f297c06eed0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f29900099d0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 msgr2=0x7f29a01e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f299802b560 tx=0x7f2998001dc0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 shutdown_connections 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01e5a80 0x7f29a01e5f00 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29a006f400 0x7f29a006f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f297c06ca10 0x7f297c06eed0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 --2- 172.21.15.88:0/2770494048 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2984006f50 0x7f29a014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:32.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 >> 172.21.15.88:0/2770494048 conn(0x7f29a00b2d50 msgr2=0x7f29a015f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:32.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 shutdown_connections 2024-04-26T20:18:32.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:32.609+0000 7f29a5600640 1 -- 172.21.15.88:0/2770494048 wait complete. 2024-04-26T20:18:33.629 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T20:18:33.629 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:33.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.823+0000 7f4cbf800640 1 Processor -- start 2024-04-26T20:18:33.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.823+0000 7f4cbf800640 1 -- start start 2024-04-26T20:18:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.824+0000 7f4cbf800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f4cb8159710 0x7f4cb8072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.824+0000 7f4cbf800640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb8159af0 con 0x7f4cb8073180 2024-04-26T20:18:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cbf800640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cb8072180 con 0x7f4cb80b3c30 2024-04-26T20:18:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cbf800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f4cb8160210 con 0x7f4cb8159710 2024-04-26T20:18:33.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cb7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f4cb8073180 0x7f4cb815f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59794/0 (socket says 172.21.15.88:59794) 2024-04-26T20:18:33.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cb7e00640 1 -- 172.21.15.88:0/1388270715 learned_addr learned my addr 172.21.15.88:0/1388270715 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:33.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cbce00640 1 --2- 172.21.15.88:0/1388270715 >> v2:172.21.15.132:3300/0 conn(0x7f4cb8159710 0x7f4cb8072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978838615 0 0) 0x7f4cb8159af0 con 0x7f4cb8073180 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.825+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ca0003520 con 0x7f4cb8073180 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1438417382 0 0) 0x7f4ca0003520 con 0x7f4cb8073180 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 >> v2:172.21.15.132:3300/0 conn(0x7f4cb8159710 msgr2=0x7f4cb8072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 --2- 172.21.15.88:0/1388270715 >> v2:172.21.15.132:3300/0 conn(0x7f4cb8159710 0x7f4cb8072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 >> v1:172.21.15.132:6789/0 conn(0x7f4cb80b3c30 legacy=0x7f4cb80738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb81604f0 con 0x7f4cb8073180 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f4ca4001670 con 0x7f4cb8073180 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cbce00640 1 --2- 172.21.15.88:0/1388270715 >> v2:172.21.15.132:3300/0 conn(0x7f4cb8159710 0x7f4cb8072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 >> v1:172.21.15.88:6789/0 conn(0x7f4cb8073180 legacy=0x7f4cb815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca00039b0 0x7f4ca0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 0x7f4ca0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ca00051c0 0x7f4ca000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8160210 con 0x7f4ca0004800 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca000c110 con 0x7f4ca00051c0 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca000c310 con 0x7f4ca00039b0 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.826+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 0x7f4ca0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cbce00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca00039b0 0x7f4ca0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cb7e00640 1 -- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca00039b0 msgr2=0x7f4ca0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca00039b0 0x7f4ca0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cb7e00640 1 -- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ca00051c0 msgr2=0x7f4ca000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ca00051c0 0x7f4ca000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cb7e00640 1 -- 172.21.15.88:0/1388270715 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca0003520 con 0x7f4ca0004800 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cbd800640 1 -- 172.21.15.88:0/1388270715 reap_dead start 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.827+0000 7f4cbce00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca00039b0 0x7f4ca0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 0x7f4ca0004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c240 tx=0x7f4ca0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3fd7ac2bd311f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4007cf0 con 0x7f4ca0004800 2024-04-26T20:18:33.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca40058d0 con 0x7f4ca0004800 2024-04-26T20:18:33.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cb7400640 1 -- 172.21.15.88:0/1388270715 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4005d20 con 0x7f4ca0004800 2024-04-26T20:18:33.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cbf800640 1 -- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 msgr2=0x7f4ca0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.828+0000 7f4cbf800640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 0x7f4ca0004c80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c240 tx=0x7f4ca0005830 comp rx=0 tx=0).stop 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.829+0000 7f4cbf800640 1 -- 172.21.15.88:0/1388270715 shutdown_connections 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.829+0000 7f4cbf800640 1 --2- 172.21.15.88:0/1388270715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ca0004800 0x7f4ca0004c80 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.829+0000 7f4cbf800640 1 -- 172.21.15.88:0/1388270715 >> 172.21.15.88:0/1388270715 conn(0x7f4cb80d0c40 msgr2=0x7f4cb80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.829+0000 7f4cbf800640 1 -- 172.21.15.88:0/1388270715 shutdown_connections 2024-04-26T20:18:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.829+0000 7f4cbf800640 1 -- 172.21.15.88:0/1388270715 wait complete. 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.830+0000 7f4cbf800640 1 Processor -- start 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.831+0000 7f4cbf800640 1 -- start start 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.831+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca0004800 0x7f4cb81ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb8156980 0x7f4cb81e8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbf800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 0x7f4cb81f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbf800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8070130 con 0x7f4cb80b1660 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbf800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb8072720 con 0x7f4cb8156980 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbf800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f4cb8072c40 con 0x7f4ca0004800 2024-04-26T20:18:33.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 0x7f4cb81f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 0x7f4cb81f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56902/0 (socket says 172.21.15.88:56902) 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 -- 172.21.15.88:0/2985632578 learned_addr learned my addr 172.21.15.88:0/2985632578 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb8156980 0x7f4cb81e8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbce00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca0004800 0x7f4cb81ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 -- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca0004800 msgr2=0x7f4cb81ea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f4ca0004800 0x7f4cb81ea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 -- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb8156980 msgr2=0x7f4cb81e8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb8156980 0x7f4cb81e8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cbd800640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb81e9340 con 0x7f4cb80b1660 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.832+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cb8156980 0x7f4cb81e8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cbd800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 0x7f4cb81f5390 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b28c0 tx=0x7f4cb000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0017070 con 0x7f4cb80b1660 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb00070d0 con 0x7f4cb80b1660 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00074c0 con 0x7f4cb80b1660 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb81e9620 con 0x7f4cb80b1660 2024-04-26T20:18:33.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.833+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb81e9be0 con 0x7f4cb80b1660 2024-04-26T20:18:33.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.834+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cb8069650 con 0x7f4cb80b1660 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.835+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f4cb001a070 con 0x7f4cb80b1660 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb4c00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4c9806ca40 0x7f4c9806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f4cb008acf0 con 0x7f4cb80b1660 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cbce00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4c9806ca40 0x7f4c9806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb4c00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7f4c98072900 0x7f4c98074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:33.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c980753d0 con 0x7f4c98072900 2024-04-26T20:18:33.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f4cb008d310 con 0x7f4cb80b1660 2024-04-26T20:18:33.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.836+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7f4c98072900 0x7f4c98074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:33.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.837+0000 7f4cbce00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4c9806ca40 0x7f4c9806ef00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8077470 tx=0x7f4cac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:33.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.837+0000 7f4cb7e00640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7f4c98072900 0x7f4c98074d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:33.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.838+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== osd.0 v2:172.21.15.88:6808/2647855198 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c980753d0 con 0x7f4c98072900 2024-04-26T20:18:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.852+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 --> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4cb80b5ab0 con 0x7f4c98072900 2024-04-26T20:18:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cb4c00640 1 -- 172.21.15.88:0/2985632578 <== osd.0 v2:172.21.15.88:6808/2647855198 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4cb80b5ab0 con 0x7f4c98072900 2024-04-26T20:18:33.853 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:33.854 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:33.854 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7f4c98072900 msgr2=0x7f4c98074d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6808/2647855198,v1:172.21.15.88:6809/2647855198] conn(0x7f4c98072900 0x7f4c98074d20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4c9806ca40 msgr2=0x7f4c9806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f4c9806ca40 0x7f4c9806ef00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8077470 tx=0x7f4cac008040 comp rx=0 tx=0).stop 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 msgr2=0x7f4cb81f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 --2- 172.21.15.88:0/2985632578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cb80b1660 0x7f4cb81f5390 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80b28c0 tx=0x7f4cb000c960 comp rx=0 tx=0).stop 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbd800640 1 -- 172.21.15.88:0/2985632578 reap_dead start 2024-04-26T20:18:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 shutdown_connections 2024-04-26T20:18:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.853+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 >> 172.21.15.88:0/2985632578 conn(0x7f4cb80d0c40 msgr2=0x7f4cb80b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.854+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 shutdown_connections 2024-04-26T20:18:33.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:33.854+0000 7f4cbf800640 1 -- 172.21.15.88:0/2985632578 wait complete. 2024-04-26T20:18:33.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T20:18:34.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.064+0000 7f112be00640 1 Processor -- start 2024-04-26T20:18:34.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.064+0000 7f112be00640 1 -- start start 2024-04-26T20:18:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.065+0000 7f112be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c146680 0x7f112c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.065+0000 7f112be00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f112c0b4b30 0x7f112c0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.065+0000 7f112be00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f112c146a60 con 0x7f112c0d0e70 2024-04-26T20:18:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.065+0000 7f112be00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f112c06e970 con 0x7f112c0b4b30 2024-04-26T20:18:34.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.065+0000 7f112be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f112c0d17f0 con 0x7f112c146680 2024-04-26T20:18:34.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.066+0000 7f112aa00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c146680 0x7f112c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.066+0000 7f112b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f112c0b4b30 0x7f112c0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.066+0000 7f112b400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f112c0b4b30 0x7f112c0d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:34122/0 (socket says 172.21.15.88:34122) 2024-04-26T20:18:34.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.066+0000 7f112b400640 1 -- 172.21.15.88:0/2519633525 learned_addr learned my addr 172.21.15.88:0/2519633525 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:34.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.067+0000 7f112b400640 1 -- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c146680 msgr2=0x7f112c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:34.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.067+0000 7f112b400640 1 --2- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c146680 0x7f112c0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.067+0000 7f112b400640 1 -- 172.21.15.88:0/2519633525 >> v1:172.21.15.132:6789/0 conn(0x7f112c0d0e70 legacy=0x7f112c160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.067+0000 7f112b400640 1 -- 172.21.15.88:0/2519633525 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c06dd60 con 0x7f112c0b4b30 2024-04-26T20:18:34.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.068+0000 7f112b400640 1 --2- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c0d12b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e50 tx=0x7f112402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68400476e500749 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.069+0000 7f1129600640 1 -- 172.21.15.88:0/2519633525 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124036070 con 0x7f112c0b4b30 2024-04-26T20:18:34.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.069+0000 7f1129600640 1 -- 172.21.15.88:0/2519633525 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1124031600 con 0x7f112c0b4b30 2024-04-26T20:18:34.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.069+0000 7f1129600640 1 -- 172.21.15.88:0/2519633525 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240319d0 con 0x7f112c0b4b30 2024-04-26T20:18:34.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 -- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 msgr2=0x7f112c0d12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 --2- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c0d12b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1124009e50 tx=0x7f112402a710 comp rx=0 tx=0).stop 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 -- 172.21.15.88:0/2519633525 shutdown_connections 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 --2- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c0d12b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 --2- 172.21.15.88:0/2519633525 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c146680 0x7f112c0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 -- 172.21.15.88:0/2519633525 >> 172.21.15.88:0/2519633525 conn(0x7f112c06c740 msgr2=0x7f112c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 -- 172.21.15.88:0/2519633525 shutdown_connections 2024-04-26T20:18:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.070+0000 7f112be00640 1 -- 172.21.15.88:0/2519633525 wait complete. 2024-04-26T20:18:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.071+0000 7f112be00640 1 Processor -- start 2024-04-26T20:18:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.071+0000 7f112be00640 1 -- start start 2024-04-26T20:18:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c13c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c0d0e70 0x7f112c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f112c06e600 con 0x7f112c146680 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f112c06e320 con 0x7f112c0b4b30 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.072+0000 7f112be00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f112c0cd6a0 con 0x7f112c0d0e70 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56910/0 (socket says 172.21.15.88:56910) 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 -- 172.21.15.88:0/2013051982 learned_addr learned my addr 172.21.15.88:0/2013051982 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 -- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c0d0e70 msgr2=0x7f112c15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c0d0e70 0x7f112c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 -- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 msgr2=0x7f112c13c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112aa00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c13c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c13c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 -- 172.21.15.88:0/2013051982 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112c13b060 con 0x7f112c146680 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.073+0000 7f112b400640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1124034040 tx=0x7f1124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.074+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124036050 con 0x7f112c146680 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.074+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1124042070 con 0x7f112c146680 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.074+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124009510 con 0x7f112c146680 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.074+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112c13b340 con 0x7f112c146680 2024-04-26T20:18:34.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.074+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112c13b900 con 0x7f112c146680 2024-04-26T20:18:34.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.075+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112c0693c0 con 0x7f112c146680 2024-04-26T20:18:34.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.076+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f1124007bf0 con 0x7f112c146680 2024-04-26T20:18:34.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.077+0000 7f1122a00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 0x7f110c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.077+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f11240ac9f0 con 0x7f112c146680 2024-04-26T20:18:34.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.077+0000 7f112aa00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 0x7f110c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.078+0000 7f112aa00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 0x7f110c06eed0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f111c009970 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.081+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1124045360 con 0x7f112c146680 2024-04-26T20:18:34.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.281+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f112c141af0 con 0x7f110c06ca10 2024-04-26T20:18:34.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.282+0000 7f1122a00640 1 -- 172.21.15.88:0/2013051982 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f112c141af0 con 0x7f110c06ca10 2024-04-26T20:18:34.282 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T20:18:34.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.284+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 msgr2=0x7f110c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.284+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 0x7f110c06eed0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f111c009970 tx=0x7f111c008040 comp rx=0 tx=0).stop 2024-04-26T20:18:34.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.284+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 msgr2=0x7f112c0cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.284+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1124034040 tx=0x7f1124004a40 comp rx=0 tx=0).stop 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 shutdown_connections 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f112c146680 0x7f112c0cbeb0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f110c06ca10 0x7f110c06eed0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f112c0d0e70 0x7f112c15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 --2- 172.21.15.88:0/2013051982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f112c0b4b30 0x7f112c13c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 >> 172.21.15.88:0/2013051982 conn(0x7f112c06c740 msgr2=0x7f112c06cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 shutdown_connections 2024-04-26T20:18:34.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.285+0000 7f112be00640 1 -- 172.21.15.88:0/2013051982 wait complete. 2024-04-26T20:18:34.302 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T20:18:34.303 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.491+0000 7fb757e00640 1 Processor -- start 2024-04-26T20:18:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.491+0000 7fb757e00640 1 -- start start 2024-04-26T20:18:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580cc810 0x7fb758160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb758162e00 0x7fb7580ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757e00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7581637e0 con 0x7fb7580c9ea0 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757e00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb75815f7c0 con 0x7fb758162e00 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb758163ac0 con 0x7fb7580cc810 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb757400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb758162e00 0x7fb7580ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb756a00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb7580c9ea0 0x7fb7580ca280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59808/0 (socket says 172.21.15.88:59808) 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.493+0000 7fb756a00640 1 -- 172.21.15.88:0/2861580613 learned_addr learned my addr 172.21.15.88:0/2861580613 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb756000640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580cc810 0x7fb758160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb757400640 1 -- 172.21.15.88:0/2861580613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580cc810 msgr2=0x7fb758160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:34.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb757400640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580cc810 0x7fb758160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb757400640 1 -- 172.21.15.88:0/2861580613 >> v1:172.21.15.88:6789/0 conn(0x7fb7580c9ea0 legacy=0x7fb7580ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb757400640 1 -- 172.21.15.88:0/2861580613 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75815fa20 con 0x7fb758162e00 2024-04-26T20:18:34.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.494+0000 7fb755600640 1 -- 172.21.15.88:0/2861580613 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384710616 0 0) 0x7fb7581637e0 con 0x7fb7580c9ea0 2024-04-26T20:18:34.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.495+0000 7fb757400640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb7580ce720 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb750009d90 tx=0x7fb75002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfed3c0d205bbae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.495+0000 7fb755600640 1 -- 172.21.15.88:0/2861580613 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750007740 con 0x7fb758162e00 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.495+0000 7fb755600640 1 -- 172.21.15.88:0/2861580613 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7500078e0 con 0x7fb758162e00 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.495+0000 7fb755600640 1 -- 172.21.15.88:0/2861580613 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750007cb0 con 0x7fb758162e00 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 -- 172.21.15.88:0/2861580613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 msgr2=0x7fb7580ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb7580ce720 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb750009d90 tx=0x7fb75002a660 comp rx=0 tx=0).stop 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 -- 172.21.15.88:0/2861580613 shutdown_connections 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb7580ce720 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 --2- 172.21.15.88:0/2861580613 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580cc810 0x7fb758160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 -- 172.21.15.88:0/2861580613 >> 172.21.15.88:0/2861580613 conn(0x7fb7580d0b50 msgr2=0x7fb7580d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 -- 172.21.15.88:0/2861580613 shutdown_connections 2024-04-26T20:18:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.496+0000 7fb757e00640 1 -- 172.21.15.88:0/2861580613 wait complete. 2024-04-26T20:18:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.498+0000 7fb757e00640 1 Processor -- start 2024-04-26T20:18:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.498+0000 7fb757e00640 1 -- start start 2024-04-26T20:18:34.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580c9ea0 0x7fb75815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb75812fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb758130300 0x7fb75806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb758145950 con 0x7fb758162e00 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb758160760 con 0x7fb758130300 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb757e00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb758145ad0 con 0x7fb7580c9ea0 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb756000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb75812fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb756000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb75812fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56932/0 (socket says 172.21.15.88:56932) 2024-04-26T20:18:34.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.499+0000 7fb756000640 1 -- 172.21.15.88:0/917028955 learned_addr learned my addr 172.21.15.88:0/917028955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756a00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580c9ea0 0x7fb75815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 -- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580c9ea0 msgr2=0x7fb75815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580c9ea0 0x7fb75815eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb757400640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb758130300 0x7fb75806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 -- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb758130300 msgr2=0x7fb75806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb758130300 0x7fb75806ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580b4910 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756a00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb7580c9ea0 0x7fb75815eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb756000640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb75812fdc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb74400d5f0 tx=0x7fb74400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440125f0 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb744019070 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440129d0 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.500+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb758130800 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.501+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7580b4bb0 con 0x7fb758162e00 2024-04-26T20:18:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.501+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb758130d20 con 0x7fb758162e00 2024-04-26T20:18:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.503+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fb744012b70 con 0x7fb758162e00 2024-04-26T20:18:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.503+0000 7fb74ea00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb73806ca10 0x7fb73806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fb74408bf50 con 0x7fb758162e00 2024-04-26T20:18:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb756a00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb73806ca10 0x7fb73806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb74ea00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fb7380728d0 0x7fb738074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7380753a0 con 0x7fb7380728d0 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb74408e570 con 0x7fb758162e00 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb757400640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fb7380728d0 0x7fb738074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.504+0000 7fb756a00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb73806ca10 0x7fb73806eed0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb748000f60 tx=0x7fb748009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.505+0000 7fb757400640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fb7380728d0 0x7fb738074cf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.506+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== osd.1 v2:172.21.15.88:6800/125701657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7380753a0 con 0x7fb7380728d0 2024-04-26T20:18:34.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.520+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 --> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb7580b5170 con 0x7fb7380728d0 2024-04-26T20:18:34.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.521+0000 7fb74ea00640 1 -- 172.21.15.88:0/917028955 <== osd.1 v2:172.21.15.88:6800/125701657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb7580b5170 con 0x7fb7380728d0 2024-04-26T20:18:34.522 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:34.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.521+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fb7380728d0 msgr2=0x7fb738074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.521+0000 7fb757e00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6800/125701657,v1:172.21.15.88:6801/125701657] conn(0x7fb7380728d0 0x7fb738074cf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb73806ca10 msgr2=0x7fb73806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb73806ca10 0x7fb73806eed0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb748000f60 tx=0x7fb748009040 comp rx=0 tx=0).stop 2024-04-26T20:18:34.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 msgr2=0x7fb75812fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 --2- 172.21.15.88:0/917028955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb758162e00 0x7fb75812fdc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb74400d5f0 tx=0x7fb74400dab0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757400640 1 -- 172.21.15.88:0/917028955 reap_dead start 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 shutdown_connections 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 >> 172.21.15.88:0/917028955 conn(0x7fb7580d0b50 msgr2=0x7fb758141da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 shutdown_connections 2024-04-26T20:18:34.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.522+0000 7fb757e00640 1 -- 172.21.15.88:0/917028955 wait complete. 2024-04-26T20:18:34.539 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T20:18:34.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.736+0000 7f1e8d000640 1 Processor -- start 2024-04-26T20:18:34.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.737+0000 7f1e8d000640 1 -- start start 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e8d000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f1e880d0e70 0x7f1e880d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e8d000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e880b3b20 0x7f1e880b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e8d000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e88146a60 con 0x7f1e88146680 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e8d000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8806e970 con 0x7f1e880b3b20 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e8d000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f1e880b60a0 con 0x7f1e880d0e70 2024-04-26T20:18:34.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e87e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e880b3b20 0x7f1e880b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e87e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f1e880b3b20 0x7f1e880b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56942/0 (socket says 172.21.15.88:56942) 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.738+0000 7f1e87e00640 1 -- 172.21.15.88:0/4229056565 learned_addr learned my addr 172.21.15.88:0/4229056565 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.739+0000 7f1e7fe00640 1 --2- 172.21.15.88:0/4229056565 >> v2:172.21.15.132:3300/0 conn(0x7f1e880d0e70 0x7f1e880d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.739+0000 7f1e87e00640 1 -- 172.21.15.88:0/4229056565 >> v2:172.21.15.132:3300/0 conn(0x7f1e880d0e70 msgr2=0x7f1e880d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.739+0000 7f1e87e00640 1 --2- 172.21.15.88:0/4229056565 >> v2:172.21.15.132:3300/0 conn(0x7f1e880d0e70 0x7f1e880d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.739+0000 7f1e87e00640 1 -- 172.21.15.88:0/4229056565 >> v1:172.21.15.132:6789/0 conn(0x7f1e88146680 legacy=0x7f1e880b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:34.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.739+0000 7f1e87e00640 1 -- 172.21.15.88:0/4229056565 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8806dd60 con 0x7f1e880b3b20 2024-04-26T20:18:34.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.740+0000 7f1e87e00640 1 --2- 172.21.15.88:0/4229056565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880b3f70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1e88160890 tx=0x7f1e8002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3744103bce350f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.740+0000 7f1e86a00640 1 -- 172.21.15.88:0/4229056565 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80036070 con 0x7f1e880b3b20 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.740+0000 7f1e86a00640 1 -- 172.21.15.88:0/4229056565 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80031600 con 0x7f1e880b3b20 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.740+0000 7f1e86a00640 1 -- 172.21.15.88:0/4229056565 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e800319d0 con 0x7f1e880b3b20 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 -- 172.21.15.88:0/4229056565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 msgr2=0x7f1e880b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 --2- 172.21.15.88:0/4229056565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880b3f70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1e88160890 tx=0x7f1e8002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 -- 172.21.15.88:0/4229056565 shutdown_connections 2024-04-26T20:18:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 --2- 172.21.15.88:0/4229056565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880b3f70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 --2- 172.21.15.88:0/4229056565 >> v2:172.21.15.132:3300/0 conn(0x7f1e880d0e70 0x7f1e880d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 -- 172.21.15.88:0/4229056565 >> 172.21.15.88:0/4229056565 conn(0x7f1e8806c740 msgr2=0x7f1e8806cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 -- 172.21.15.88:0/4229056565 shutdown_connections 2024-04-26T20:18:34.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.741+0000 7f1e8d000640 1 -- 172.21.15.88:0/4229056565 wait complete. 2024-04-26T20:18:34.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.743+0000 7f1e8d000640 1 Processor -- start 2024-04-26T20:18:34.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.743+0000 7f1e8d000640 1 -- start start 2024-04-26T20:18:34.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e880d0e70 0x7f1e880c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e88146680 0x7f1e881f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8806e320 con 0x7f1e880b3b20 2024-04-26T20:18:34.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880d17f0 con 0x7f1e88146680 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e8d000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8806e600 con 0x7f1e880d0e70 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e87400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e87400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56946/0 (socket says 172.21.15.88:56946) 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.744+0000 7f1e87400640 1 -- 172.21.15.88:0/2766303248 learned_addr learned my addr 172.21.15.88:0/2766303248 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e7fe00640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e880d0e70 0x7f1e880c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 -- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e880d0e70 msgr2=0x7f1e880c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e880d0e70 0x7f1e880c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 -- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e88146680 msgr2=0x7f1e881f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e88146680 0x7f1e881f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 -- 172.21.15.88:0/2766303248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880ca3e0 con 0x7f1e880b3b20 2024-04-26T20:18:34.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e87400640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b3f0 tx=0x7f1e7800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78015020 con 0x7f1e880b3b20 2024-04-26T20:18:34.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e78011830 con 0x7f1e880b3b20 2024-04-26T20:18:34.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8813c120 con 0x7f1e880b3b20 2024-04-26T20:18:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78011c20 con 0x7f1e880b3b20 2024-04-26T20:18:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.745+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8813c6e0 con 0x7f1e880b3b20 2024-04-26T20:18:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.747+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e880693c0 con 0x7f1e880b3b20 2024-04-26T20:18:34.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.748+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f1e78018020 con 0x7f1e880b3b20 2024-04-26T20:18:34.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.748+0000 7f1e7f400640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 0x7f1e6406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:34.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.749+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f1e7808b590 con 0x7f1e880b3b20 2024-04-26T20:18:34.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.749+0000 7f1e7fe00640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 0x7f1e6406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:34.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.749+0000 7f1e7fe00640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 0x7f1e6406ef00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e881f83e0 tx=0x7f1e70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:34.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.753+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e7801f030 con 0x7f1e880b3b20 2024-04-26T20:18:34.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.952+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f1e880cc940 con 0x7f1e6406ca40 2024-04-26T20:18:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.952+0000 7f1e7f400640 1 -- 172.21.15.88:0/2766303248 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1e70009d60 con 0x7f1e6406ca40 2024-04-26T20:18:34.953 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 msgr2=0x7f1e6406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 0x7f1e6406ef00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e881f83e0 tx=0x7f1e70008040 comp rx=0 tx=0).stop 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 msgr2=0x7f1e880cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b3f0 tx=0x7f1e7800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 shutdown_connections 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e88146680 0x7f1e881f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f1e6406ca40 0x7f1e6406ef00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f1e880d0e70 0x7f1e880c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 --2- 172.21.15.88:0/2766303248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e880b3b20 0x7f1e880cc400 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 >> 172.21.15.88:0/2766303248 conn(0x7f1e8806c740 msgr2=0x7f1e88160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 shutdown_connections 2024-04-26T20:18:34.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:34.955+0000 7f1e8d000640 1 -- 172.21.15.88:0/2766303248 wait complete. 2024-04-26T20:18:34.974 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T20:18:34.975 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.166+0000 7f34ef200640 1 Processor -- start 2024-04-26T20:18:35.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.166+0000 7f34ef200640 1 -- start start 2024-04-26T20:18:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.167+0000 7f34ef200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f34e806f9d0 0x7f34e806fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.167+0000 7f34ef200640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e80b5f40 con 0x7f34e815bf50 2024-04-26T20:18:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.167+0000 7f34ef200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e815c330 con 0x7f34e80b5210 2024-04-26T20:18:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.167+0000 7f34ef200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f34e8070370 con 0x7f34e806f9d0 2024-04-26T20:18:35.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e7e00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f34e815bf50 0x7f34e8072030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59820/0 (socket says 172.21.15.88:59820) 2024-04-26T20:18:35.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e7e00640 1 -- 172.21.15.88:0/1903274748 learned_addr learned my addr 172.21.15.88:0/1903274748 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:35.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e7400640 1 --2- 172.21.15.88:0/1903274748 >> v2:172.21.15.132:3300/0 conn(0x7f34e806f9d0 0x7f34e806fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2966293972 0 0) 0x7f34e80b5f40 con 0x7f34e815bf50 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34cc003520 con 0x7f34e815bf50 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978955884 0 0) 0x7f34cc003520 con 0x7f34e815bf50 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.168+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 >> v2:172.21.15.132:3300/0 conn(0x7f34e806f9d0 msgr2=0x7f34e806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 --2- 172.21.15.88:0/1903274748 >> v2:172.21.15.132:3300/0 conn(0x7f34e806f9d0 0x7f34e806fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 >> v1:172.21.15.132:6789/0 conn(0x7f34e80b5210 legacy=0x7f34e80b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e80aeaa0 con 0x7f34e815bf50 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f34d8001670 con 0x7f34e815bf50 2024-04-26T20:18:35.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 >> v1:172.21.15.88:6789/0 conn(0x7f34e815bf50 legacy=0x7f34e8072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc0039b0 0x7f34cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 0x7f34cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc0051c0 0x7f34cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00c030 con 0x7f34cc004800 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc00c1e0 con 0x7f34cc0051c0 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc00c3e0 con 0x7f34cc0039b0 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e7400640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 0x7f34cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc0039b0 0x7f34cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34ed200640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc0051c0 0x7f34cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.169+0000 7f34e7400640 1 -- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc0039b0 msgr2=0x7f34cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34e7400640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc0039b0 0x7f34cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34e7400640 1 -- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc0051c0 msgr2=0x7f34cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34e7400640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc0051c0 0x7f34cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34e7400640 1 -- 172.21.15.88:0/1903274748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc003520 con 0x7f34cc004800 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34cc0039b0 0x7f34cc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:35.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34ed200640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc0051c0 0x7f34cc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:35.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.170+0000 7f34ed200640 1 -- 172.21.15.88:0/1903274748 reap_dead start 2024-04-26T20:18:35.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34e7400640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 0x7f34cc004c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc46435fa0ffd2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4036070 con 0x7f34cc004800 2024-04-26T20:18:35.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d4031400 con 0x7f34cc004800 2024-04-26T20:18:35.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34e6a00640 1 -- 172.21.15.88:0/1903274748 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4009690 con 0x7f34cc004800 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34ef200640 1 -- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 msgr2=0x7f34cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.171+0000 7f34ef200640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 0x7f34cc004c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34cc005830 comp rx=0 tx=0).stop 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.172+0000 7f34ef200640 1 -- 172.21.15.88:0/1903274748 shutdown_connections 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.172+0000 7f34ef200640 1 --2- 172.21.15.88:0/1903274748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34cc004800 0x7f34cc004c80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.172+0000 7f34ef200640 1 -- 172.21.15.88:0/1903274748 >> 172.21.15.88:0/1903274748 conn(0x7f34e80d0c40 msgr2=0x7f34e80d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.172+0000 7f34ef200640 1 -- 172.21.15.88:0/1903274748 shutdown_connections 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.172+0000 7f34ef200640 1 -- 172.21.15.88:0/1903274748 wait complete. 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.173+0000 7f34ef200640 1 Processor -- start 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 -- start start 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc004800 0x7f34e809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34e809e930 0x7f34e8156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 0x7f34e80cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34e80af340 con 0x7f34e8157390 2024-04-26T20:18:35.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f34e80af060 con 0x7f34cc004800 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.174+0000 7f34ef200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f34e806fe30 con 0x7f34e809e930 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 0x7f34e80cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 0x7f34e80cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56966/0 (socket says 172.21.15.88:56966) 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 -- 172.21.15.88:0/1847079503 learned_addr learned my addr 172.21.15.88:0/1847079503 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34e7400640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34e809e930 0x7f34e8156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc004800 0x7f34e809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 -- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34e809e930 msgr2=0x7f34e8156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f34e809e930 0x7f34e8156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 -- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc004800 msgr2=0x7f34e809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc004800 0x7f34e809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34ed200640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e81f4c70 con 0x7f34e8157390 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.175+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34cc004800 0x7f34e809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34ed200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 0x7f34e80cf220 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34e8074860 tx=0x7f34e000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0018070 con 0x7f34e8157390 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e0003da0 con 0x7f34e8157390 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0006d90 con 0x7f34e8157390 2024-04-26T20:18:35.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e81f4ef0 con 0x7f34e8157390 2024-04-26T20:18:35.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.176+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81f54b0 con 0x7f34e8157390 2024-04-26T20:18:35.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.177+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34e8069650 con 0x7f34e8157390 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.178+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f34e0005070 con 0x7f34e8157390 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34dfe00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f34c806c9e0 0x7f34c806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f34e008ac60 con 0x7f34e8157390 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f34c806c9e0 0x7f34c806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34dfe00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f34c80728a0 0x7f34c8074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34e001be50 con 0x7f34c80728a0 2024-04-26T20:18:35.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.179+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f34e008f2a0 con 0x7f34e8157390 2024-04-26T20:18:35.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.180+0000 7f34e7400640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f34c80728a0 0x7f34c8074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.180+0000 7f34e7e00640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f34c806c9e0 0x7f34c806eea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34e8077470 tx=0x7f34d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.180+0000 7f34e7400640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f34c80728a0 0x7f34c8074cc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.181+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== osd.2 v2:172.21.15.88:6821/3051227728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34e001be50 con 0x7f34c80728a0 2024-04-26T20:18:35.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.196+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 --> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f34e806e340 con 0x7f34c80728a0 2024-04-26T20:18:35.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.196+0000 7f34dfe00640 1 -- 172.21.15.88:0/1847079503 <== osd.2 v2:172.21.15.88:6821/3051227728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f34e806e340 con 0x7f34c80728a0 2024-04-26T20:18:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:35.197 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:35.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f34c80728a0 msgr2=0x7f34c8074cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6821/3051227728,v1:172.21.15.88:6823/3051227728] conn(0x7f34c80728a0 0x7f34c8074cc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f34c806c9e0 msgr2=0x7f34c806eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f34c806c9e0 0x7f34c806eea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34e8077470 tx=0x7f34d8009040 comp rx=0 tx=0).stop 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 msgr2=0x7f34e80cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 --2- 172.21.15.88:0/1847079503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34e8157390 0x7f34e80cf220 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f34e8074860 tx=0x7f34e000d9d0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ed200640 1 -- 172.21.15.88:0/1847079503 reap_dead start 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 shutdown_connections 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 >> 172.21.15.88:0/1847079503 conn(0x7f34e80d0c40 msgr2=0x7f34e80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 shutdown_connections 2024-04-26T20:18:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.197+0000 7f34ef200640 1 -- 172.21.15.88:0/1847079503 wait complete. 2024-04-26T20:18:35.215 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T20:18:35.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.416+0000 7fb638a00640 1 Processor -- start 2024-04-26T20:18:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.416+0000 7fb638a00640 1 -- start start 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.417+0000 7fb638a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb6340d1580 0x7fb63415f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.417+0000 7fb638a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb63415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.417+0000 7fb638a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb634073110 con 0x7fb634145570 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.417+0000 7fb638a00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fb6340cef40 con 0x7fb6340d1580 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.417+0000 7fb638a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb6341606b0 con 0x7fb634073ed0 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632a00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fb6340d1580 0x7fb63415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb633400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fb634145570 0x7fb6340d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59832/0 (socket says 172.21.15.88:59832) 2024-04-26T20:18:35.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb633400640 1 -- 172.21.15.88:0/2313235916 learned_addr learned my addr 172.21.15.88:0/2313235916 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:35.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb633e00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb63415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632a00640 1 -- 172.21.15.88:0/2313235916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 msgr2=0x7fb63415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632a00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb63415fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632a00640 1 -- 172.21.15.88:0/2313235916 >> v1:172.21.15.88:6789/0 conn(0x7fb634145570 legacy=0x7fb6340d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632a00640 1 -- 172.21.15.88:0/2313235916 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340733f0 con 0x7fb6340d1580 2024-04-26T20:18:35.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.418+0000 7fb632000640 1 -- 172.21.15.88:0/2313235916 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175576542 0 0) 0x7fb634073110 con 0x7fb634145570 2024-04-26T20:18:35.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.419+0000 7fb632a00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb63415f940 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb6200097a0 tx=0x7fb62002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5779220a46bdb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb632000640 1 -- 172.21.15.88:0/2313235916 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620037070 con 0x7fb6340d1580 2024-04-26T20:18:35.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb632000640 1 -- 172.21.15.88:0/2313235916 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62002ae50 con 0x7fb6340d1580 2024-04-26T20:18:35.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb632000640 1 -- 172.21.15.88:0/2313235916 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620032660 con 0x7fb6340d1580 2024-04-26T20:18:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 -- 172.21.15.88:0/2313235916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 msgr2=0x7fb63415f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb63415f940 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb6200097a0 tx=0x7fb62002a100 comp rx=0 tx=0).stop 2024-04-26T20:18:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 -- 172.21.15.88:0/2313235916 shutdown_connections 2024-04-26T20:18:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb63415fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 --2- 172.21.15.88:0/2313235916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb63415f940 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 -- 172.21.15.88:0/2313235916 >> 172.21.15.88:0/2313235916 conn(0x7fb63406c740 msgr2=0x7fb63406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 -- 172.21.15.88:0/2313235916 shutdown_connections 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.420+0000 7fb638a00640 1 -- 172.21.15.88:0/2313235916 wait complete. 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.422+0000 7fb638a00640 1 Processor -- start 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.422+0000 7fb638a00640 1 -- start start 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb6340cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb634145570 0x7fb634142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341603c0 con 0x7fb6340d1580 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340739b0 con 0x7fb634145570 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb638a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fb634145950 con 0x7fb634073ed0 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.423+0000 7fb632a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56974/0 (socket says 172.21.15.88:56974) 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 -- 172.21.15.88:0/715252855 learned_addr learned my addr 172.21.15.88:0/715252855 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 -- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 msgr2=0x7fb6340cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb6340cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 -- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb634145570 msgr2=0x7fb634142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb634145570 0x7fb634142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 -- 172.21.15.88:0/715252855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb634133800 con 0x7fb6340d1580 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.424+0000 7fb632a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098c0 tx=0x7fb6200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.425+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620041030 con 0x7fb6340d1580 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.425+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62002ac50 con 0x7fb6340d1580 2024-04-26T20:18:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.425+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620037040 con 0x7fb6340d1580 2024-04-26T20:18:35.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.425+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6340cce40 con 0x7fb6340d1580 2024-04-26T20:18:35.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.425+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6340ca950 con 0x7fb6340d1580 2024-04-26T20:18:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.426+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6340693c0 con 0x7fb6340d1580 2024-04-26T20:18:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.427+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fb620044070 con 0x7fb6340d1580 2024-04-26T20:18:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.428+0000 7fb62b400640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 0x7fb61406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.428+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fb6200ac060 con 0x7fb6340d1580 2024-04-26T20:18:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.428+0000 7fb633400640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 0x7fb61406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.428+0000 7fb633400640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 0x7fb61406ef00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb624007fd0 tx=0x7fb624007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.432+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb62007a700 con 0x7fb6340d1580 2024-04-26T20:18:35.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.632+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fb6340765d0 con 0x7fb61406ca40 2024-04-26T20:18:35.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.633+0000 7fb62b400640 1 -- 172.21.15.88:0/715252855 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb6340765d0 con 0x7fb61406ca40 2024-04-26T20:18:35.634 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 msgr2=0x7fb61406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 0x7fb61406ef00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb624007fd0 tx=0x7fb624007960 comp rx=0 tx=0).stop 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 msgr2=0x7fb6341332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098c0 tx=0x7fb6200046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 shutdown_connections 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb634145570 0x7fb634142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6340d1580 0x7fb6341332c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fb61406ca40 0x7fb61406ef00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 --2- 172.21.15.88:0/715252855 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fb634073ed0 0x7fb6340cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 >> 172.21.15.88:0/715252855 conn(0x7fb63406c740 msgr2=0x7fb6340cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 shutdown_connections 2024-04-26T20:18:35.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.636+0000 7fb638a00640 1 -- 172.21.15.88:0/715252855 wait complete. 2024-04-26T20:18:35.655 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T20:18:35.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:35.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.857+0000 7f3c49800640 1 Processor -- start 2024-04-26T20:18:35.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.857+0000 7f3c49800640 1 -- start start 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f3c44159710 0x7f3c44072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 0x7f3c4415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3c4415fc20 0x7f3c440b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f3c44159af0 con 0x7f3c4415fc20 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f3c44072180 con 0x7f3c44159710 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c49800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3c44160210 con 0x7f3c44073180 2024-04-26T20:18:35.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c43400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3c4415fc20 0x7f3c440b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c43400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f3c4415fc20 0x7f3c440b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56982/0 (socket says 172.21.15.88:56982) 2024-04-26T20:18:35.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.859+0000 7f3c43400640 1 -- 172.21.15.88:0/4197686455 learned_addr learned my addr 172.21.15.88:0/4197686455 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c42a00640 1 --2- 172.21.15.88:0/4197686455 >> v2:172.21.15.132:3300/0 conn(0x7f3c44159710 0x7f3c44072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c3be00640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 0x7f3c4415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c43400640 1 -- 172.21.15.88:0/4197686455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 msgr2=0x7f3c4415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c43400640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 0x7f3c4415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c43400640 1 -- 172.21.15.88:0/4197686455 >> v2:172.21.15.132:3300/0 conn(0x7f3c44159710 msgr2=0x7f3c44072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c43400640 1 --2- 172.21.15.88:0/4197686455 >> v2:172.21.15.132:3300/0 conn(0x7f3c44159710 0x7f3c44072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c43400640 1 -- 172.21.15.88:0/4197686455 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441604f0 con 0x7f3c4415fc20 2024-04-26T20:18:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.860+0000 7f3c3be00640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 0x7f3c4415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.861+0000 7f3c43400640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4415fc20 0x7f3c440b3c30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3c44072150 tx=0x7f3c3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe2f5e5fb71817f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.861+0000 7f3c42000640 1 -- 172.21.15.88:0/4197686455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c036070 con 0x7f3c4415fc20 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.861+0000 7f3c42000640 1 -- 172.21.15.88:0/4197686455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3c031600 con 0x7f3c4415fc20 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.861+0000 7f3c42000640 1 -- 172.21.15.88:0/4197686455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0319d0 con 0x7f3c4415fc20 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 -- 172.21.15.88:0/4197686455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4415fc20 msgr2=0x7f3c440b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4415fc20 0x7f3c440b3c30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3c44072150 tx=0x7f3c3c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 -- 172.21.15.88:0/4197686455 shutdown_connections 2024-04-26T20:18:35.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c4415fc20 0x7f3c440b3c30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 --2- 172.21.15.88:0/4197686455 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44073180 0x7f3c4415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 --2- 172.21.15.88:0/4197686455 >> v2:172.21.15.132:3300/0 conn(0x7f3c44159710 0x7f3c44072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 -- 172.21.15.88:0/4197686455 >> 172.21.15.88:0/4197686455 conn(0x7f3c440d0c40 msgr2=0x7f3c440d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 -- 172.21.15.88:0/4197686455 shutdown_connections 2024-04-26T20:18:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.862+0000 7f3c49800640 1 -- 172.21.15.88:0/4197686455 wait complete. 2024-04-26T20:18:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.864+0000 7f3c49800640 1 Processor -- start 2024-04-26T20:18:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.864+0000 7f3c49800640 1 -- start start 2024-04-26T20:18:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 0x7f3c440af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44159710 0x7f3c44156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c4415fc20 0x7f3c4414c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44070130 con 0x7f3c44073180 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44072720 con 0x7f3c4415fc20 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c49800640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f3c440b4170 con 0x7f3c44159710 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c42a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 0x7f3c440af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c42a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 0x7f3c440af820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56984/0 (socket says 172.21.15.88:56984) 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.865+0000 7f3c42a00640 1 -- 172.21.15.88:0/3497297798 learned_addr learned my addr 172.21.15.88:0/3497297798 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 -- 172.21.15.88:0/3497297798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44159710 msgr2=0x7f3c44156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f3c44159710 0x7f3c44156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 -- 172.21.15.88:0/3497297798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c4415fc20 msgr2=0x7f3c4414c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c4415fc20 0x7f3c4414c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441554b0 con 0x7f3c44073180 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c42a00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 0x7f3c440af820 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d800 tx=0x7f3c3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400b6c0 con 0x7f3c44073180 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c34019070 con 0x7f3c44073180 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3400ba80 con 0x7f3c44073180 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c44155790 con 0x7f3c44073180 2024-04-26T20:18:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.866+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c440af300 con 0x7f3c44073180 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.867+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c44069650 con 0x7f3c44073180 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.868+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f3c3400bc20 con 0x7f3c44073180 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.869+0000 7f3c3aa00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3c2006ca10 0x7f3c2006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.869+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f3c3408bee0 con 0x7f3c44073180 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c3aa00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f3c200728d0 0x7f3c20074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:35.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c3be00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3c2006ca10 0x7f3c2006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c34001d30 con 0x7f3c200728d0 2024-04-26T20:18:35.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3c3408e500 con 0x7f3c44073180 2024-04-26T20:18:35.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c43400640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f3c200728d0 0x7f3c20074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:35.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c3be00640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3c2006ca10 0x7f3c2006eed0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3c4414db60 tx=0x7f3c2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.870+0000 7f3c43400640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f3c200728d0 0x7f3c20074cf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:35.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.871+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== osd.3 v2:172.21.15.88:6816/1302198393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c34001d30 con 0x7f3c200728d0 2024-04-26T20:18:35.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.886+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 --> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c4406e8d0 con 0x7f3c200728d0 2024-04-26T20:18:35.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.886+0000 7f3c3aa00640 1 -- 172.21.15.88:0/3497297798 <== osd.3 v2:172.21.15.88:6816/1302198393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3c4406e8d0 con 0x7f3c200728d0 2024-04-26T20:18:35.887 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:35.887 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:35.887 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.886+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f3c200728d0 msgr2=0x7f3c20074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.886+0000 7f3c49800640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6816/1302198393,v1:172.21.15.88:6817/1302198393] conn(0x7f3c200728d0 0x7f3c20074cf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3c2006ca10 msgr2=0x7f3c2006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f3c2006ca10 0x7f3c2006eed0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3c4414db60 tx=0x7f3c2c008040 comp rx=0 tx=0).stop 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 msgr2=0x7f3c440af820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 --2- 172.21.15.88:0/3497297798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c44073180 0x7f3c440af820 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d800 tx=0x7f3c3400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c43400640 1 -- 172.21.15.88:0/3497297798 reap_dead start 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 shutdown_connections 2024-04-26T20:18:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 >> 172.21.15.88:0/3497297798 conn(0x7f3c440d0c40 msgr2=0x7f3c44163570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:35.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 shutdown_connections 2024-04-26T20:18:35.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:35.887+0000 7f3c49800640 1 -- 172.21.15.88:0/3497297798 wait complete. 2024-04-26T20:18:35.904 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T20:18:36.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.101+0000 7f1736600640 1 Processor -- start 2024-04-26T20:18:36.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.101+0000 7f1736600640 1 -- start start 2024-04-26T20:18:36.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.102+0000 7f1736600640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f17301598a0 0x7f17301638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.102+0000 7f1736600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17300b5fb0 con 0x7f173015fe50 2024-04-26T20:18:36.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f1736600640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1730160370 con 0x7f173006c770 2024-04-26T20:18:36.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f1736600640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f1730159c80 con 0x7f17301598a0 2024-04-26T20:18:36.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172f400640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f17301598a0 0x7f17301638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172fe00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f173015fe50 0x7f17300add80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59846/0 (socket says 172.21.15.88:59846) 2024-04-26T20:18:36.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172fe00640 1 -- 172.21.15.88:0/1982220588 learned_addr learned my addr 172.21.15.88:0/1982220588 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:36.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172f400640 1 -- 172.21.15.88:0/1982220588 >> v1:172.21.15.132:6789/0 conn(0x7f173006c770 legacy=0x7f173015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:36.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172f400640 1 -- 172.21.15.88:0/1982220588 >> v1:172.21.15.88:6789/0 conn(0x7f173015fe50 legacy=0x7f17300add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.103+0000 7f172f400640 1 -- 172.21.15.88:0/1982220588 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1730162dd0 con 0x7f17301598a0 2024-04-26T20:18:36.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.104+0000 7f172f400640 1 --2- 172.21.15.88:0/1982220588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f17301638b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f17200097a0 tx=0x7f172002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f805bec5224c0623 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f172ea00640 1 -- 172.21.15.88:0/1982220588 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720037070 con 0x7f17301598a0 2024-04-26T20:18:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f172ea00640 1 -- 172.21.15.88:0/1982220588 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172002ae50 con 0x7f17301598a0 2024-04-26T20:18:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f172ea00640 1 -- 172.21.15.88:0/1982220588 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720032660 con 0x7f17301598a0 2024-04-26T20:18:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 -- 172.21.15.88:0/1982220588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 msgr2=0x7f17301638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 --2- 172.21.15.88:0/1982220588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f17301638b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f17200097a0 tx=0x7f172002a140 comp rx=0 tx=0).stop 2024-04-26T20:18:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 -- 172.21.15.88:0/1982220588 shutdown_connections 2024-04-26T20:18:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 --2- 172.21.15.88:0/1982220588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f17301638b0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 -- 172.21.15.88:0/1982220588 >> 172.21.15.88:0/1982220588 conn(0x7f17300b2d50 msgr2=0x7f17300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:36.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.105+0000 7f1736600640 1 -- 172.21.15.88:0/1982220588 shutdown_connections 2024-04-26T20:18:36.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.106+0000 7f1736600640 1 -- 172.21.15.88:0/1982220588 wait complete. 2024-04-26T20:18:36.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.107+0000 7f1736600640 1 Processor -- start 2024-04-26T20:18:36.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.107+0000 7f1736600640 1 -- start start 2024-04-26T20:18:36.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f173006c770 0x7f17301444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173015fe50 0x7f17300b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1730160650 con 0x7f17301598a0 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f1730163390 con 0x7f173015fe50 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1736600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f173006cbd0 con 0x7f173006c770 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.108+0000 7f1726a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56992/0 (socket says 172.21.15.88:56992) 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 -- 172.21.15.88:0/2173219013 learned_addr learned my addr 172.21.15.88:0/2173219013 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 -- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f173006c770 msgr2=0x7f17301444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f172f400640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f173006c770 0x7f17301444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f173006c770 0x7f17301444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 -- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173015fe50 msgr2=0x7f17300b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f172fe00640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173015fe50 0x7f17300b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173015fe50 0x7f17300b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 -- 172.21.15.88:0/2173219013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17300cea00 con 0x7f17301598a0 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.109+0000 7f1726a00640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f171800c430 tx=0x7f171800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.110+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718011790 con 0x7f17301598a0 2024-04-26T20:18:36.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.110+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1718017070 con 0x7f17301598a0 2024-04-26T20:18:36.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.110+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718011bb0 con 0x7f17301598a0 2024-04-26T20:18:36.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.110+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17300cece0 con 0x7f17301598a0 2024-04-26T20:18:36.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.110+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17300cf2a0 con 0x7f17301598a0 2024-04-26T20:18:36.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.111+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1730069540 con 0x7f17301598a0 2024-04-26T20:18:36.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.112+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f171801c460 con 0x7f17301598a0 2024-04-26T20:18:36.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.113+0000 7f1727e00640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 0x7f170c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.113+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f171808b990 con 0x7f17301598a0 2024-04-26T20:18:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.113+0000 7f172f400640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 0x7f170c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.114+0000 7f172f400640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 0x7f170c06eed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1720034000 tx=0x7f172002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.117+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1718059f10 con 0x7f17301598a0 2024-04-26T20:18:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.318+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f17300adeb0 con 0x7f170c06ca10 2024-04-26T20:18:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.319+0000 7f1727e00640 1 -- 172.21.15.88:0/2173219013 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f17300adeb0 con 0x7f170c06ca10 2024-04-26T20:18:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T20:18:36.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 msgr2=0x7f170c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 0x7f170c06eed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1720034000 tx=0x7f172002e000 comp rx=0 tx=0).stop 2024-04-26T20:18:36.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 msgr2=0x7f173006fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f171800c430 tx=0x7f171800c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 shutdown_connections 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f173015fe50 0x7f17300b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17301598a0 0x7f173006fbe0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f170c06ca10 0x7f170c06eed0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 --2- 172.21.15.88:0/2173219013 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f173006c770 0x7f17301444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.321+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 >> 172.21.15.88:0/2173219013 conn(0x7f17300b2d50 msgr2=0x7f17300d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.322+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 shutdown_connections 2024-04-26T20:18:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.322+0000 7f1736600640 1 -- 172.21.15.88:0/2173219013 wait complete. 2024-04-26T20:18:36.341 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T20:18:36.341 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:36.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.534+0000 7f564bc00640 1 Processor -- start 2024-04-26T20:18:36.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.535+0000 7f564bc00640 1 -- start start 2024-04-26T20:18:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.536+0000 7f564bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f5644072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.536+0000 7f564bc00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56440b3c30 0x7f56440b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f564bc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5644159af0 con 0x7f5644073180 2024-04-26T20:18:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f564bc00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f5644072180 con 0x7f56440b3c30 2024-04-26T20:18:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f564bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f5644160210 con 0x7f5644159710 2024-04-26T20:18:36.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f5649c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56440b3c30 0x7f56440b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f5649200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f5644072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f5649c00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f56440b3c30 0x7f56440b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:34192/0 (socket says 172.21.15.88:34192) 2024-04-26T20:18:36.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.537+0000 7f5649c00640 1 -- 172.21.15.88:0/2158640520 learned_addr learned my addr 172.21.15.88:0/2158640520 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:36.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.538+0000 7f5649c00640 1 -- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 msgr2=0x7f5644072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:36.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.538+0000 7f5649c00640 1 --2- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f5644072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.538+0000 7f5649c00640 1 -- 172.21.15.88:0/2158640520 >> v1:172.21.15.132:6789/0 conn(0x7f5644073180 legacy=0x7f564415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.538+0000 7f5649c00640 1 -- 172.21.15.88:0/2158640520 --> v2:172.21.15.132:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441604f0 con 0x7f56440b3c30 2024-04-26T20:18:36.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.540+0000 7f5649c00640 1 --2- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b4080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f563c009e50 tx=0x7f563c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe0c5c43c302e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.540+0000 7f5643e00640 1 -- 172.21.15.88:0/2158640520 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c036070 con 0x7f56440b3c30 2024-04-26T20:18:36.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.540+0000 7f5643e00640 1 -- 172.21.15.88:0/2158640520 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563c031600 con 0x7f56440b3c30 2024-04-26T20:18:36.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.540+0000 7f5643e00640 1 -- 172.21.15.88:0/2158640520 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c0319d0 con 0x7f56440b3c30 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 -- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 msgr2=0x7f56440b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 --2- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b4080 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f563c009e50 tx=0x7f563c02a710 comp rx=0 tx=0).stop 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 -- 172.21.15.88:0/2158640520 shutdown_connections 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 --2- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b4080 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 --2- 172.21.15.88:0/2158640520 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f5644072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 -- 172.21.15.88:0/2158640520 >> 172.21.15.88:0/2158640520 conn(0x7f56440d0c40 msgr2=0x7f56440d1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:36.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 -- 172.21.15.88:0/2158640520 shutdown_connections 2024-04-26T20:18:36.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.541+0000 7f564bc00640 1 -- 172.21.15.88:0/2158640520 wait complete. 2024-04-26T20:18:36.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.543+0000 7f564bc00640 1 Processor -- start 2024-04-26T20:18:36.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.543+0000 7f564bc00640 1 -- start start 2024-04-26T20:18:36.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 0x7f564409d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f564414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5644070130 con 0x7f5644073180 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f5644072720 con 0x7f56440b3c30 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f564bc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f564415fdf0 con 0x7f5644159710 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f5649200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 0x7f564409d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f5649200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 0x7f564409d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57004/0 (socket says 172.21.15.88:57004) 2024-04-26T20:18:36.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.544+0000 7f5649200640 1 -- 172.21.15.88:0/132716310 learned_addr learned my addr 172.21.15.88:0/132716310 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649c00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f564414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 -- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 msgr2=0x7f564414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f563be00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f564414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 -- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 msgr2=0x7f56440b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56440b3c30 0x7f56440b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564414fcf0 con 0x7f5644073180 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649c00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f5644159710 0x7f564414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:36.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5649200640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 0x7f564409d050 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f563400e9e0 tx=0x7f563400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634017070 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f563400c9d0 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563400cdc0 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.545+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56440b1ac0 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.546+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56440b2080 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.546+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5644069650 con 0x7f5644073180 2024-04-26T20:18:36.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.548+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f563401a070 con 0x7f5644073180 2024-04-26T20:18:36.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.548+0000 7f5641600640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f562006ca10 0x7f562006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f563408bdb0 con 0x7f5644073180 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f563be00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f562006ca10 0x7f562006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f5641600640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f56200728d0 0x7f5620074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56200753a0 con 0x7f56200728d0 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f563408e3d0 con 0x7f5644073180 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.549+0000 7f563be00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f562006ca10 0x7f562006eed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f564409eaa0 tx=0x7f562c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.550+0000 7f5649c00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f56200728d0 0x7f5620074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.550+0000 7f5649c00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f56200728d0 0x7f5620074cf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.552+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== osd.4 v2:172.21.15.132:6816/861564477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56200753a0 con 0x7f56200728d0 2024-04-26T20:18:36.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.567+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 --> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f56440b24a0 con 0x7f56200728d0 2024-04-26T20:18:36.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f5641600640 1 -- 172.21.15.88:0/132716310 <== osd.4 v2:172.21.15.132:6816/861564477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f56440b24a0 con 0x7f56200728d0 2024-04-26T20:18:36.569 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:36.569 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:36.569 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f56200728d0 msgr2=0x7f5620074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.132:6816/861564477,v1:172.21.15.132:6818/861564477] conn(0x7f56200728d0 0x7f5620074cf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f562006ca10 msgr2=0x7f562006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f562006ca10 0x7f562006eed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f564409eaa0 tx=0x7f562c009040 comp rx=0 tx=0).stop 2024-04-26T20:18:36.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 msgr2=0x7f564409d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.568+0000 7f564bc00640 1 --2- 172.21.15.88:0/132716310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5644073180 0x7f564409d050 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f563400e9e0 tx=0x7f563400eea0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.569+0000 7f5649c00640 1 -- 172.21.15.88:0/132716310 reap_dead start 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.569+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 shutdown_connections 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.569+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 >> 172.21.15.88:0/132716310 conn(0x7f56440d0c40 msgr2=0x7f5644163020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.569+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 shutdown_connections 2024-04-26T20:18:36.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.569+0000 7f564bc00640 1 -- 172.21.15.88:0/132716310 wait complete. 2024-04-26T20:18:36.588 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T20:18:36.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.785+0000 7fe238000640 1 Processor -- start 2024-04-26T20:18:36.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.786+0000 7fe238000640 1 -- start start 2024-04-26T20:18:36.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.787+0000 7fe238000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fe23006ce80 0x7fe2300add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.787+0000 7fe238000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe2300cd650 0x7fe2300b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.787+0000 7fe238000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2300af9a0 con 0x7fe23015cdf0 2024-04-26T20:18:36.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.787+0000 7fe238000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fe2300ae2c0 con 0x7fe2300cd650 2024-04-26T20:18:36.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.787+0000 7fe238000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fe2300b05b0 con 0x7fe23006ce80 2024-04-26T20:18:36.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe236000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fe2300cd650 0x7fe2300b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe235600640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fe23015cdf0 0x7fe23006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59858/0 (socket says 172.21.15.88:59858) 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe235600640 1 -- 172.21.15.88:0/1224823076 learned_addr learned my addr 172.21.15.88:0/1224823076 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe234c00640 1 --2- 172.21.15.88:0/1224823076 >> v2:172.21.15.132:3300/0 conn(0x7fe23006ce80 0x7fe2300add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe236000640 1 -- 172.21.15.88:0/1224823076 >> v2:172.21.15.132:3300/0 conn(0x7fe23006ce80 msgr2=0x7fe2300add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe236000640 1 --2- 172.21.15.88:0/1224823076 >> v2:172.21.15.132:3300/0 conn(0x7fe23006ce80 0x7fe2300add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe236000640 1 -- 172.21.15.88:0/1224823076 >> v1:172.21.15.88:6789/0 conn(0x7fe23015cdf0 legacy=0x7fe23006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe236000640 1 -- 172.21.15.88:0/1224823076 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2300afc80 con 0x7fe2300cd650 2024-04-26T20:18:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.788+0000 7fe22fe00640 1 -- 172.21.15.88:0/1224823076 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672506770 0 0) 0x7fe2300af9a0 con 0x7fe23015cdf0 2024-04-26T20:18:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.789+0000 7fe236000640 1 --2- 172.21.15.88:0/1224823076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe2300b3d10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2300d1d00 tx=0x7fe22802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca933abde5a52c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe22fe00640 1 -- 172.21.15.88:0/1224823076 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228037070 con 0x7fe2300cd650 2024-04-26T20:18:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe22fe00640 1 -- 172.21.15.88:0/1224823076 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228032440 con 0x7fe2300cd650 2024-04-26T20:18:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe22fe00640 1 -- 172.21.15.88:0/1224823076 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228032810 con 0x7fe2300cd650 2024-04-26T20:18:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 -- 172.21.15.88:0/1224823076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 msgr2=0x7fe2300b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 --2- 172.21.15.88:0/1224823076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe2300b3d10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2300d1d00 tx=0x7fe22802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 -- 172.21.15.88:0/1224823076 shutdown_connections 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 --2- 172.21.15.88:0/1224823076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe2300b3d10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 --2- 172.21.15.88:0/1224823076 >> v2:172.21.15.132:3300/0 conn(0x7fe23006ce80 0x7fe2300add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 -- 172.21.15.88:0/1224823076 >> 172.21.15.88:0/1224823076 conn(0x7fe2300b2d50 msgr2=0x7fe2300b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 -- 172.21.15.88:0/1224823076 shutdown_connections 2024-04-26T20:18:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.790+0000 7fe238000640 1 -- 172.21.15.88:0/1224823076 wait complete. 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.792+0000 7fe238000640 1 Processor -- start 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.792+0000 7fe238000640 1 -- start start 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe23006ce80 0x7fe2301568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe23015cdf0 0x7fe2300cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe23015d1d0 con 0x7fe2300cd650 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fe2300b0240 con 0x7fe23006ce80 2024-04-26T20:18:36.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.793+0000 7fe238000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fe23015d350 con 0x7fe23015cdf0 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57020/0 (socket says 172.21.15.88:57020) 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 -- 172.21.15.88:0/2277307768 learned_addr learned my addr 172.21.15.88:0/2277307768 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe235600640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe23006ce80 0x7fe2301568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe236000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe23015cdf0 0x7fe2300cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 -- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe23015cdf0 msgr2=0x7fe2300cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe23015cdf0 0x7fe2300cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 -- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe23006ce80 msgr2=0x7fe2301568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe23006ce80 0x7fe2301568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 -- 172.21.15.88:0/2277307768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23006f300 con 0x7fe2300cd650 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.794+0000 7fe234c00640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d800 tx=0x7fe21c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.795+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00b6c0 con 0x7fe2300cd650 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.795+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c019070 con 0x7fe2300cd650 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.795+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23006ea60 con 0x7fe2300cd650 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.795+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00ba80 con 0x7fe2300cd650 2024-04-26T20:18:36.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.795+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23006f020 con 0x7fe2300cd650 2024-04-26T20:18:36.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.796+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe230069540 con 0x7fe2300cd650 2024-04-26T20:18:36.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.797+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fe21c00bc20 con 0x7fe2300cd650 2024-04-26T20:18:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.798+0000 7fe22d600640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 0x7fe21006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.798+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fe21c08be90 con 0x7fe2300cd650 2024-04-26T20:18:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.798+0000 7fe235600640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 0x7fe21006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:36.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.799+0000 7fe235600640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 0x7fe21006eed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2240012e0 tx=0x7fe224009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:36.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:36.802+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe21c05a4b0 con 0x7fe2300cd650 2024-04-26T20:18:37.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.000+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe23015d530 con 0x7fe21006ca10 2024-04-26T20:18:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.001+0000 7fe22d600640 1 -- 172.21.15.88:0/2277307768 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe23015d530 con 0x7fe21006ca10 2024-04-26T20:18:37.002 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 msgr2=0x7fe21006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 0x7fe21006eed0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe2240012e0 tx=0x7fe224009040 comp rx=0 tx=0).stop 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 msgr2=0x7fe23006e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d800 tx=0x7fe21c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 shutdown_connections 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fe21006ca10 0x7fe21006eed0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fe23015cdf0 0x7fe2300cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2300cd650 0x7fe23006e010 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 --2- 172.21.15.88:0/2277307768 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe23006ce80 0x7fe2301568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 >> 172.21.15.88:0/2277307768 conn(0x7fe2300b2d50 msgr2=0x7fe23006c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 shutdown_connections 2024-04-26T20:18:37.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.004+0000 7fe238000640 1 -- 172.21.15.88:0/2277307768 wait complete. 2024-04-26T20:18:37.024 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T20:18:37.024 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:37.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.230+0000 7fc812c00640 1 Processor -- start 2024-04-26T20:18:37.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.231+0000 7fc812c00640 1 -- start start 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812c00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7fc80c0cb8b0 0x7fc80c0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812c00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80c06df60 con 0x7fc80c143470 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812c00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7fc80c146ba0 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c06e200 con 0x7fc80c143850 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc811800640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fc80c143470 0x7fc80c0cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59870/0 (socket says 172.21.15.88:59870) 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc811800640 1 -- 172.21.15.88:0/3500780574 learned_addr learned my addr 172.21.15.88:0/3500780574 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.232+0000 7fc812200640 1 --2- 172.21.15.88:0/3500780574 >> v2:172.21.15.88:3300/0 conn(0x7fc80c0cb8b0 0x7fc80c0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc80be00640 1 -- 172.21.15.88:0/3500780574 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326534825 0 0) 0x7fc80c06df60 con 0x7fc80c143470 2024-04-26T20:18:37.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc80be00640 1 -- 172.21.15.88:0/3500780574 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f4003520 con 0x7fc80c143470 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc810e00640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc812200640 1 -- 172.21.15.88:0/3500780574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 msgr2=0x7fc80c06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc812200640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c06fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc812200640 1 -- 172.21.15.88:0/3500780574 >> v1:172.21.15.88:6789/0 conn(0x7fc80c143470 legacy=0x7fc80c0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc812200640 1 -- 172.21.15.88:0/3500780574 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c06e4e0 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.233+0000 7fc810e00640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c06fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.234+0000 7fc812200640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c0723c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530e8a4372f5ddce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.234+0000 7fc80be00640 1 -- 172.21.15.88:0/3500780574 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804031600 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.234+0000 7fc80be00640 1 -- 172.21.15.88:0/3500780574 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8040317a0 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.234+0000 7fc80be00640 1 -- 172.21.15.88:0/3500780574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804031b50 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 -- 172.21.15.88:0/3500780574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c0cb8b0 msgr2=0x7fc80c0723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c0723c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fc804009d90 tx=0x7fc80402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 -- 172.21.15.88:0/3500780574 shutdown_connections 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c0723c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 --2- 172.21.15.88:0/3500780574 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c06fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 -- 172.21.15.88:0/3500780574 >> 172.21.15.88:0/3500780574 conn(0x7fc80c0d0b50 msgr2=0x7fc80c0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 -- 172.21.15.88:0/3500780574 shutdown_connections 2024-04-26T20:18:37.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.235+0000 7fc812c00640 1 -- 172.21.15.88:0/3500780574 wait complete. 2024-04-26T20:18:37.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.237+0000 7fc812c00640 1 Processor -- start 2024-04-26T20:18:37.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.237+0000 7fc812c00640 1 -- start start 2024-04-26T20:18:37.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c1f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 0x7fc80c12eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c1e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0cbd00 con 0x7fc80c143470 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c070180 con 0x7fc80c0cb8b0 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc812c00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c0cbe80 con 0x7fc80c143850 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc810e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 0x7fc80c12eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc810e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 0x7fc80c12eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57034/0 (socket says 172.21.15.88:57034) 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.238+0000 7fc810e00640 1 -- 172.21.15.88:0/4284938852 learned_addr learned my addr 172.21.15.88:0/4284938852 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc811800640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c1f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc812200640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c1e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 -- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 msgr2=0x7fc80c1e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c1e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 -- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c0cb8b0 msgr2=0x7fc80c1f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc80c0cb8b0 0x7fc80c1f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c140310 con 0x7fc80c143470 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc812200640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fc80c143850 0x7fc80c1e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:37.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc810e00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 0x7fc80c12eea0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c7f0 tx=0x7fc7f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8015070 con 0x7fc80c143470 2024-04-26T20:18:37.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.239+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8007a30 con 0x7fc80c143470 2024-04-26T20:18:37.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.240+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1405f0 con 0x7fc80c143470 2024-04-26T20:18:37.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.240+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8007e00 con 0x7fc80c143470 2024-04-26T20:18:37.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.240+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1f5560 con 0x7fc80c143470 2024-04-26T20:18:37.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.240+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc80c06d1f0 con 0x7fc80c143470 2024-04-26T20:18:37.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.242+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fc7f8018070 con 0x7fc80c143470 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc809600640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc7ec06ca10 0x7fc7ec06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fc7f808bb40 con 0x7fc80c143470 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc811800640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc7ec06ca10 0x7fc7ec06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc809600640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fc7ec0728d0 0x7fc7ec074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7ec0753a0 con 0x7fc7ec0728d0 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc7f80902a0 con 0x7fc80c143470 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.243+0000 7fc811800640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc7ec06ca10 0x7fc7ec06eed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc800007fd0 tx=0x7fc800009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.244+0000 7fc812200640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fc7ec0728d0 0x7fc7ec074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.245+0000 7fc812200640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fc7ec0728d0 0x7fc7ec074cf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.247+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== osd.5 v2:172.21.15.132:6817/4023647186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7ec0753a0 con 0x7fc7ec0728d0 2024-04-26T20:18:37.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.262+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 --> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc80c0693c0 con 0x7fc7ec0728d0 2024-04-26T20:18:37.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc809600640 1 -- 172.21.15.88:0/4284938852 <== osd.5 v2:172.21.15.132:6817/4023647186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc80c0693c0 con 0x7fc7ec0728d0 2024-04-26T20:18:37.263 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:37.263 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:37.264 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fc7ec0728d0 msgr2=0x7fc7ec074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc812c00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.132:6817/4023647186,v1:172.21.15.132:6819/4023647186] conn(0x7fc7ec0728d0 0x7fc7ec074cf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc7ec06ca10 msgr2=0x7fc7ec06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc812c00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fc7ec06ca10 0x7fc7ec06eed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc800007fd0 tx=0x7fc800009040 comp rx=0 tx=0).stop 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.263+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 msgr2=0x7fc80c12eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812c00640 1 --2- 172.21.15.88:0/4284938852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc80c143470 0x7fc80c12eea0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c7f0 tx=0x7fc7f800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812200640 1 -- 172.21.15.88:0/4284938852 reap_dead start 2024-04-26T20:18:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 shutdown_connections 2024-04-26T20:18:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 >> 172.21.15.88:0/4284938852 conn(0x7fc80c0d0b50 msgr2=0x7fc80c0b5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 shutdown_connections 2024-04-26T20:18:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.264+0000 7fc812c00640 1 -- 172.21.15.88:0/4284938852 wait complete. 2024-04-26T20:18:37.281 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T20:18:37.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.473+0000 7fd314a00640 1 Processor -- start 2024-04-26T20:18:37.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.473+0000 7fd314a00640 1 -- start start 2024-04-26T20:18:37.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd314a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd3100b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd314a00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd310146a60 con 0x7fd3100b4b30 2024-04-26T20:18:37.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd314a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd310161750 con 0x7fd3100d0e70 2024-04-26T20:18:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd314a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd31006e970 con 0x7fd310146680 2024-04-26T20:18:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd30f200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fd3100b4b30 0x7fd310161040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59884/0 (socket says 172.21.15.88:59884) 2024-04-26T20:18:37.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd30f200640 1 -- 172.21.15.88:0/2452948134 learned_addr learned my addr 172.21.15.88:0/2452948134 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd30e800640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd3100b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.475+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198458885 0 0) 0x7fd310146a60 con 0x7fd3100b4b30 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2f4003520 con 0x7fd3100b4b30 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4042135518 0 0) 0x7fd2f4003520 con 0x7fd3100b4b30 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 msgr2=0x7fd3100b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd3100b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 >> v1:172.21.15.132:6789/0 conn(0x7fd3100d0e70 legacy=0x7fd310160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31006dd60 con 0x7fd3100b4b30 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fd3080019f0 con 0x7fd3100b4b30 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 >> v1:172.21.15.88:6789/0 conn(0x7fd3100b4b30 legacy=0x7fd310161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f40039e0 0x7fd2f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd2f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd2f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400bb50 con 0x7fd2f4004d40 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f400bcd0 con 0x7fd2f40039e0 2024-04-26T20:18:37.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.476+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f400bea0 con 0x7fd310146680 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd2f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30de00640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd2f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30e800640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f40039e0 0x7fd2f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 -- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 msgr2=0x7fd2f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd310146680 0x7fd2f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 -- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f40039e0 msgr2=0x7fd2f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2f40039e0 0x7fd2f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.477+0000 7fd30f200640 1 -- 172.21.15.88:0/2452948134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f4003520 con 0x7fd2f4004d40 2024-04-26T20:18:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.478+0000 7fd30f200640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd2f40051c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd30800a860 tx=0x7fd2f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08f817d78c16913 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.478+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30800a480 con 0x7fd2f4004d40 2024-04-26T20:18:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.478+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30802f070 con 0x7fd2f4004d40 2024-04-26T20:18:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.478+0000 7fd30d400640 1 -- 172.21.15.88:0/2452948134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30802bca0 con 0x7fd2f4004d40 2024-04-26T20:18:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 -- 172.21.15.88:0/2452948134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 msgr2=0x7fd2f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 --2- 172.21.15.88:0/2452948134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd2f40051c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd30800a860 tx=0x7fd2f4005830 comp rx=0 tx=0).stop 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd30f200640 1 -- 172.21.15.88:0/2452948134 reap_dead start 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 -- 172.21.15.88:0/2452948134 shutdown_connections 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 -- 172.21.15.88:0/2452948134 >> 172.21.15.88:0/2452948134 conn(0x7fd31006c740 msgr2=0x7fd31006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 -- 172.21.15.88:0/2452948134 shutdown_connections 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.479+0000 7fd314a00640 1 -- 172.21.15.88:0/2452948134 wait complete. 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.481+0000 7fd314a00640 1 Processor -- start 2024-04-26T20:18:37.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.481+0000 7fd314a00640 1 -- start start 2024-04-26T20:18:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100c9ea0 0x7fd3100ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100cbc30 0x7fd3100cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd31006e600 con 0x7fd2f4004d40 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fd31006e320 con 0x7fd3100c9ea0 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd314a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fd3100cd6a0 con 0x7fd3100cbc30 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd30e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd30e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57054/0 (socket says 172.21.15.88:57054) 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.482+0000 7fd30e800640 1 -- 172.21.15.88:0/1685121210 learned_addr learned my addr 172.21.15.88:0/1685121210 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:37.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 -- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100cbc30 msgr2=0x7fd3100cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100cbc30 0x7fd3100cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 -- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100c9ea0 msgr2=0x7fd3100ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100c9ea0 0x7fd3100ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 -- 172.21.15.88:0/1685121210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31013dac0 con 0x7fd2f4004d40 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd30e800640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd30000f5e0 tx=0x7fd30000faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300019020 con 0x7fd2f4004d40 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300014870 con 0x7fd2f4004d40 2024-04-26T20:18:37.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.483+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd310150270 con 0x7fd2f4004d40 2024-04-26T20:18:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.484+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300014c60 con 0x7fd2f4004d40 2024-04-26T20:18:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.484+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd310150830 con 0x7fd2f4004d40 2024-04-26T20:18:37.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.485+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3100693c0 con 0x7fd2f4004d40 2024-04-26T20:18:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.486+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fd30001c020 con 0x7fd2f4004d40 2024-04-26T20:18:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.487+0000 7fd306a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 0x7fd2f006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.487+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fd30008c190 con 0x7fd2f4004d40 2024-04-26T20:18:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.487+0000 7fd30de00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 0x7fd2f006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.488+0000 7fd30de00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 0x7fd2f006eed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.492+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd300023020 con 0x7fd2f4004d40 2024-04-26T20:18:37.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.698+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fd3100d1500 con 0x7fd2f006ca10 2024-04-26T20:18:37.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.698+0000 7fd306a00640 1 -- 172.21.15.88:0/1685121210 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd3100d1500 con 0x7fd2f006ca10 2024-04-26T20:18:37.699 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T20:18:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 msgr2=0x7fd2f006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 0x7fd2f006eed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc007fd0 tx=0x7fd2fc009040 comp rx=0 tx=0).stop 2024-04-26T20:18:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 msgr2=0x7fd31013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd30000f5e0 tx=0x7fd30000faa0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 shutdown_connections 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fd2f006ca10 0x7fd2f006eed0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fd3100cbc30 0x7fd3100cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3100c9ea0 0x7fd3100ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 --2- 172.21.15.88:0/1685121210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f4004d40 0x7fd31013d580 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 >> 172.21.15.88:0/1685121210 conn(0x7fd31006c740 msgr2=0x7fd3100ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 shutdown_connections 2024-04-26T20:18:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.701+0000 7fd314a00640 1 -- 172.21.15.88:0/1685121210 wait complete. 2024-04-26T20:18:37.720 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T20:18:37.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:37.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.916+0000 7f692a000640 1 Processor -- start 2024-04-26T20:18:37.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.916+0000 7f692a000640 1 -- start start 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f692a000640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f69240d2140 0x7f69240d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f692a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 0x7f69240d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f692a000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69240d2e60 con 0x7f692415f1c0 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f692a000640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f69240d3a70 con 0x7f69240d2140 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f692a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f69240d3140 con 0x7f69240d3f00 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f6923e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f69240d2140 0x7f69240d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f6923e00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f69240d2140 0x7f69240d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57070/0 (socket says 172.21.15.88:57070) 2024-04-26T20:18:37.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.918+0000 7f6923e00640 1 -- 172.21.15.88:0/2017400304 learned_addr learned my addr 172.21.15.88:0/2017400304 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:37.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6929600640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 0x7f69240d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6923e00640 1 -- 172.21.15.88:0/2017400304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 msgr2=0x7f69240d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6923e00640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 0x7f69240d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6923e00640 1 -- 172.21.15.88:0/2017400304 >> v1:172.21.15.132:6789/0 conn(0x7f692415f1c0 legacy=0x7f69240d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6923e00640 1 -- 172.21.15.88:0/2017400304 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240d3420 con 0x7f69240d2140 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.919+0000 7f6929600640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 0x7f69240d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:37.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.920+0000 7f6923e00640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69240d2140 0x7f69240d0760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e50 tx=0x7f690c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6098a70c09d43b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.920+0000 7f6923400640 1 -- 172.21.15.88:0/2017400304 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c036070 con 0x7f69240d2140 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f6923400640 1 -- 172.21.15.88:0/2017400304 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c031600 con 0x7f69240d2140 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f6923400640 1 -- 172.21.15.88:0/2017400304 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0319d0 con 0x7f69240d2140 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 -- 172.21.15.88:0/2017400304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69240d2140 msgr2=0x7f69240d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69240d2140 0x7f69240d0760 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e50 tx=0x7f690c02a6d0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 -- 172.21.15.88:0/2017400304 shutdown_connections 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d3f00 0x7f69240d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 --2- 172.21.15.88:0/2017400304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69240d2140 0x7f69240d0760 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 -- 172.21.15.88:0/2017400304 >> 172.21.15.88:0/2017400304 conn(0x7f692409e340 msgr2=0x7f69240d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 -- 172.21.15.88:0/2017400304 shutdown_connections 2024-04-26T20:18:37.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.921+0000 7f692a000640 1 -- 172.21.15.88:0/2017400304 wait complete. 2024-04-26T20:18:37.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.923+0000 7f692a000640 1 Processor -- start 2024-04-26T20:18:37.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.923+0000 7f692a000640 1 -- start start 2024-04-26T20:18:37.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d2140 0x7f6924150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69240d3f00 0x7f69240c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 0x7f69240c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f692415f720 con 0x7f692415f1c0 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f692415f5a0 con 0x7f69240d3f00 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f692a000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f69240b48a0 con 0x7f69240d2140 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f6929600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 0x7f69240c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f6929600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 0x7f69240c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57084/0 (socket says 172.21.15.88:57084) 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.924+0000 7f6929600640 1 -- 172.21.15.88:0/882703237 learned_addr learned my addr 172.21.15.88:0/882703237 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 -- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d2140 msgr2=0x7f6924150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6923e00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69240d3f00 0x7f69240c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6928c00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d2140 0x7f6924150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f69240d2140 0x7f6924150640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 -- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69240d3f00 msgr2=0x7f69240c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69240d3f00 0x7f69240c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924161e10 con 0x7f692415f1c0 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6923e00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69240d3f00 0x7f69240c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6929600640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 0x7f69240c95f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c820 tx=0x7f691c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.925+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c015070 con 0x7f692415f1c0 2024-04-26T20:18:37.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.926+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c007a30 con 0x7f692415f1c0 2024-04-26T20:18:37.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.926+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c007e20 con 0x7f692415f1c0 2024-04-26T20:18:37.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.926+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6924161570 con 0x7f692415f1c0 2024-04-26T20:18:37.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.926+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6924161b30 con 0x7f692415f1c0 2024-04-26T20:18:37.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.927+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69240693c0 con 0x7f692415f1c0 2024-04-26T20:18:37.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.928+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f691c018070 con 0x7f692415f1c0 2024-04-26T20:18:37.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6920c00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f690006ca10 0x7f690006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f691c08ba10 con 0x7f692415f1c0 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6928c00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f690006ca10 0x7f690006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6920c00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f69000728d0 0x7f6900074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f691c01de50 con 0x7f69000728d0 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.929+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f691c0902a0 con 0x7f692415f1c0 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.930+0000 7f6928c00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f690006ca10 0x7f690006eed0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f691400a750 tx=0x7f6914008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.930+0000 7f6923e00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f69000728d0 0x7f6900074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:37.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.931+0000 7f6923e00640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f69000728d0 0x7f6900074cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:37.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.933+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== osd.6 v2:172.21.15.132:6800/2163437982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f691c01de50 con 0x7f69000728d0 2024-04-26T20:18:37.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.947+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 --> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f69240d26e0 con 0x7f69000728d0 2024-04-26T20:18:37.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.948+0000 7f6920c00640 1 -- 172.21.15.88:0/882703237 <== osd.6 v2:172.21.15.132:6800/2163437982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f69240d26e0 con 0x7f69000728d0 2024-04-26T20:18:37.949 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:37.949 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:37.949 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:37.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f69000728d0 msgr2=0x7f6900074cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.132:6800/2163437982,v1:172.21.15.132:6801/2163437982] conn(0x7f69000728d0 0x7f6900074cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f690006ca10 msgr2=0x7f690006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f690006ca10 0x7f690006eed0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f691400a750 tx=0x7f6914008040 comp rx=0 tx=0).stop 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 msgr2=0x7f69240c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 --2- 172.21.15.88:0/882703237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692415f1c0 0x7f69240c95f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c820 tx=0x7f691c00cce0 comp rx=0 tx=0).stop 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f6929600640 1 -- 172.21.15.88:0/882703237 reap_dead start 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 shutdown_connections 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 >> 172.21.15.88:0/882703237 conn(0x7f692409e340 msgr2=0x7f692406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 shutdown_connections 2024-04-26T20:18:37.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:37.949+0000 7f692a000640 1 -- 172.21.15.88:0/882703237 wait complete. 2024-04-26T20:18:37.966 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T20:18:38.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.161+0000 7f2a3e000640 1 Processor -- start 2024-04-26T20:18:38.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.161+0000 7f2a3e000640 1 -- start start 2024-04-26T20:18:38.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 0x7f2a380add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a3e000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a380af9a0 con 0x7f2a3806ce80 2024-04-26T20:18:38.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a3e000640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a380afc80 con 0x7f2a3815cdf0 2024-04-26T20:18:38.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a3e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2a380ae2c0 con 0x7f2a380cd650 2024-04-26T20:18:38.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a2ea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f2a3806ce80 0x7f2a380b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59898/0 (socket says 172.21.15.88:59898) 2024-04-26T20:18:38.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.163+0000 7f2a2ea00640 1 -- 172.21.15.88:0/628064711 learned_addr learned my addr 172.21.15.88:0/628064711 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860332408 0 0) 0x7f2a380af9a0 con 0x7f2a3806ce80 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a37e00640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 0x7f2a380add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a1c003520 con 0x7f2a3806ce80 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3935289285 0 0) 0x7f2a1c003520 con 0x7f2a3806ce80 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 msgr2=0x7f2a380add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 0x7f2a380add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 >> v1:172.21.15.132:6789/0 conn(0x7f2a3815cdf0 legacy=0x7f2a3806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a380aff60 con 0x7f2a3806ce80 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f2a20001670 con 0x7f2a3806ce80 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 >> v1:172.21.15.88:6789/0 conn(0x7f2a3806ce80 legacy=0x7f2a380b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c0039e0 0x7f2a1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 0x7f2a1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c004d40 0x7f2a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00bba0 con 0x7f2a1c0039e0 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c00bd50 con 0x7f2a1c004d40 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.164+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1c00bf50 con 0x7f2a380cd650 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c0039e0 0x7f2a1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 -- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 msgr2=0x7f2a1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a380cd650 0x7f2a1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 -- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c004d40 msgr2=0x7f2a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c004d40 0x7f2a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.165+0000 7f2a37400640 1 -- 172.21.15.88:0/628064711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c003520 con 0x7f2a1c0039e0 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.166+0000 7f2a37400640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c0039e0 0x7f2a1c003e00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2a28001750 tx=0x7f2a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dcd4713a8e4926 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.166+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28001370 con 0x7f2a1c0039e0 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.166+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a280052d0 con 0x7f2a1c0039e0 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.166+0000 7f2a36a00640 1 -- 172.21.15.88:0/628064711 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2802abe0 con 0x7f2a1c0039e0 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 -- 172.21.15.88:0/628064711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c0039e0 msgr2=0x7f2a1c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 --2- 172.21.15.88:0/628064711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c0039e0 0x7f2a1c003e00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2a28001750 tx=0x7f2a1c005830 comp rx=0 tx=0).stop 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a37e00640 1 -- 172.21.15.88:0/628064711 reap_dead start 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 -- 172.21.15.88:0/628064711 shutdown_connections 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 -- 172.21.15.88:0/628064711 >> 172.21.15.88:0/628064711 conn(0x7f2a380b2d50 msgr2=0x7f2a380b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 -- 172.21.15.88:0/628064711 shutdown_connections 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.167+0000 7f2a3e000640 1 -- 172.21.15.88:0/628064711 wait complete. 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.169+0000 7f2a3e000640 1 Processor -- start 2024-04-26T20:18:38.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.169+0000 7f2a3e000640 1 -- start start 2024-04-26T20:18:38.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 0x7f2a3806f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3814f580 0x7f2a3815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3815d1d0 con 0x7f2a3814fa00 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f2a380b05b0 con 0x7f2a1c0039e0 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a3e000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3815d350 con 0x7f2a3814f580 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a37e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a37e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57106/0 (socket says 172.21.15.88:57106) 2024-04-26T20:18:38.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.170+0000 7f2a37e00640 1 -- 172.21.15.88:0/3872503127 learned_addr learned my addr 172.21.15.88:0/3872503127 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 -- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3814f580 msgr2=0x7f2a3815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3814f580 0x7f2a3815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37400640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 0x7f2a3806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 -- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 msgr2=0x7f2a3806f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 0x7f2a3806f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 -- 172.21.15.88:0/3872503127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3806e3f0 con 0x7f2a3814fa00 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37400640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 0x7f2a3806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:38.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a37e00640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2a38073540 tx=0x7f2a3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007e40 con 0x7f2a3814fa00 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30019070 con 0x7f2a3814fa00 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30014660 con 0x7f2a3814fa00 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.171+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3806e6d0 con 0x7f2a3814fa00 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.172+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3806ec90 con 0x7f2a3814fa00 2024-04-26T20:18:38.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.173+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a38069540 con 0x7f2a3814fa00 2024-04-26T20:18:38.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.174+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f2a30014800 con 0x7f2a3814fa00 2024-04-26T20:18:38.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.175+0000 7f2a2fe00640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 0x7f2a1406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.175+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f2a3008be30 con 0x7f2a3814fa00 2024-04-26T20:18:38.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.175+0000 7f2a37400640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 0x7f2a1406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.176+0000 7f2a37400640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 0x7f2a1406eed0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a28032000 tx=0x7f2a2802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.179+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a3005a440 con 0x7f2a3814fa00 2024-04-26T20:18:38.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.380+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f2a380ae100 con 0x7f2a1406ca10 2024-04-26T20:18:38.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.381+0000 7f2a2fe00640 1 -- 172.21.15.88:0/3872503127 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2a380ae100 con 0x7f2a1406ca10 2024-04-26T20:18:38.381 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 msgr2=0x7f2a1406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 0x7f2a1406eed0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2a28032000 tx=0x7f2a2802e000 comp rx=0 tx=0).stop 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 msgr2=0x7f2a3815b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2a38073540 tx=0x7f2a3000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 shutdown_connections 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a3814fa00 0x7f2a3815b300 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f2a1406ca10 0x7f2a1406eed0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f2a3814f580 0x7f2a3815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3872503127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c0039e0 0x7f2a3806f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 >> 172.21.15.88:0/3872503127 conn(0x7f2a380b2d50 msgr2=0x7f2a380b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:38.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 shutdown_connections 2024-04-26T20:18:38.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.384+0000 7f2a3e000640 1 -- 172.21.15.88:0/3872503127 wait complete. 2024-04-26T20:18:38.403 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T20:18:38.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:18:38.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.601+0000 7f9308600640 1 Processor -- start 2024-04-26T20:18:38.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.601+0000 7f9308600640 1 -- start start 2024-04-26T20:18:38.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9308600640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f930015fa60 0x7f93000cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9308600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 0x7f93000cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9308600640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93000d0810 con 0x7f930015f680 2024-04-26T20:18:38.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9308600640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f93000b45b0 con 0x7f930015fa60 2024-04-26T20:18:38.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9308600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f93000cd550 con 0x7f93000cfe90 2024-04-26T20:18:38.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9305c00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f930015f680 0x7f93000cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:59910/0 (socket says 172.21.15.88:59910) 2024-04-26T20:18:38.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9305c00640 1 -- 172.21.15.88:0/1843720827 learned_addr learned my addr 172.21.15.88:0/1843720827 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:38.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9306600640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 0x7f93000cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f9305200640 1 --2- 172.21.15.88:0/1843720827 >> v2:172.21.15.132:3300/0 conn(0x7f930015fa60 0x7f93000cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.603+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239874671 0 0) 0x7f93000d0810 con 0x7f930015f680 2024-04-26T20:18:38.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92e8003520 con 0x7f930015f680 2024-04-26T20:18:38.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 789167967 0 0) 0x7f92e8003520 con 0x7f930015f680 2024-04-26T20:18:38.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 msgr2=0x7f93000cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 0x7f93000cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 >> v2:172.21.15.132:3300/0 conn(0x7f930015fa60 msgr2=0x7f93000cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 --2- 172.21.15.88:0/1843720827 >> v2:172.21.15.132:3300/0 conn(0x7f930015fa60 0x7f93000cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93000d1e10 con 0x7f930015f680 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f92f4001670 con 0x7f930015f680 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 >> v1:172.21.15.88:6789/0 conn(0x7f930015f680 legacy=0x7f93000cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92e80039b0 0x7f92e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e8004800 0x7f92e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 0x7f92e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93000b45b0 con 0x7f92e8004800 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f93000cd550 con 0x7f92e80039b0 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f92e800b9f0 con 0x7f93000cfe90 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.604+0000 7f9305200640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e8004800 0x7f92e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305c00640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92e80039b0 0x7f92e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305200640 1 -- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 msgr2=0x7f92e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305200640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f93000cfe90 0x7f92e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305200640 1 -- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92e80039b0 msgr2=0x7f92e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305200640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92e80039b0 0x7f92e8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.605+0000 7f9305200640 1 -- 172.21.15.88:0/1843720827 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e8003520 con 0x7f92e8004800 2024-04-26T20:18:38.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.606+0000 7f9305200640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e8004800 0x7f92e8004c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00ba00 tx=0x7f92e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20a374531490d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.606+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec030070 con 0x7f92e8004800 2024-04-26T20:18:38.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.606+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec02ade0 con 0x7f92e8004800 2024-04-26T20:18:38.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.606+0000 7f92ffe00640 1 -- 172.21.15.88:0/1843720827 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00b600 con 0x7f92e8004800 2024-04-26T20:18:38.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 -- 172.21.15.88:0/1843720827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e8004800 msgr2=0x7f92e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 --2- 172.21.15.88:0/1843720827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92e8004800 0x7f92e8004c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00ba00 tx=0x7f92e8005830 comp rx=0 tx=0).stop 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9306600640 1 -- 172.21.15.88:0/1843720827 reap_dead start 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 -- 172.21.15.88:0/1843720827 shutdown_connections 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 -- 172.21.15.88:0/1843720827 >> 172.21.15.88:0/1843720827 conn(0x7f93000d2f00 msgr2=0x7f93000d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 -- 172.21.15.88:0/1843720827 shutdown_connections 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.607+0000 7f9308600640 1 -- 172.21.15.88:0/1843720827 wait complete. 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.609+0000 7f9308600640 1 Processor -- start 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.609+0000 7f9308600640 1 -- start start 2024-04-26T20:18:38.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92e8004800 0x7f93000b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 0x7f930009db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f930009f8e0 0x7f930009fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93000b4280 con 0x7f930009d6c0 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f93000d23d0 con 0x7f930009f8e0 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9308600640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f9300075f80 con 0x7f92e8004800 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 0x7f930009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 0x7f930009db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57122/0 (socket says 172.21.15.88:57122) 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 -- 172.21.15.88:0/2124712128 learned_addr learned my addr 172.21.15.88:0/2124712128 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305c00640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92e8004800 0x7f93000b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 -- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92e8004800 msgr2=0x7f93000b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f92e8004800 0x7f93000b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 -- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f930009f8e0 msgr2=0x7f930009fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9306600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f930009f8e0 0x7f930009fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.610+0000 7f9305200640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f930009f8e0 0x7f930009fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f9305200640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93000a0640 con 0x7f930009d6c0 2024-04-26T20:18:38.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f9305200640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 0x7f930009db40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a8d0 tx=0x7f92ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec043020 con 0x7f930009d6c0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ec03f680 con 0x7f930009d6c0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec00b600 con 0x7f930009d6c0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93000a0920 con 0x7f930009d6c0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.611+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93000b3a10 con 0x7f930009d6c0 2024-04-26T20:18:38.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.612+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93000affa0 con 0x7f930009d6c0 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.613+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f92ec046020 con 0x7f930009d6c0 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.614+0000 7f92fd600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f92e006ca40 0x7f92e006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.614+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f92ec0ac940 con 0x7f930009d6c0 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.614+0000 7f92fd600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f92e0072900 0x7f92e0074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.615+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92ec04de80 con 0x7f92e0072900 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.615+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f92ec0b12a0 con 0x7f930009d6c0 2024-04-26T20:18:38.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.615+0000 7f9305c00640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f92e006ca40 0x7f92e006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.615+0000 7f9306600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f92e0072900 0x7f92e0074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.615+0000 7f9305c00640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f92e006ca40 0x7f92e006ef00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92f40017e0 tx=0x7f92f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.616+0000 7f9306600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f92e0072900 0x7f92e0074d20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.618+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== osd.7 v2:172.21.15.132:6808/3391944263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92ec04de80 con 0x7f92e0072900 2024-04-26T20:18:38.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.633+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 --> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9300069650 con 0x7f92e0072900 2024-04-26T20:18:38.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f92fd600640 1 -- 172.21.15.88:0/2124712128 <== osd.7 v2:172.21.15.132:6808/3391944263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9300069650 con 0x7f92e0072900 2024-04-26T20:18:38.634 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-26T20:18:38.635 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:18:38.635 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f92e0072900 msgr2=0x7f92e0074d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.132:6808/3391944263,v1:172.21.15.132:6809/3391944263] conn(0x7f92e0072900 0x7f92e0074d20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f92e006ca40 msgr2=0x7f92e006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f92e006ca40 0x7f92e006ef00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92f40017e0 tx=0x7f92f4009040 comp rx=0 tx=0).stop 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 msgr2=0x7f930009db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9308600640 1 --2- 172.21.15.88:0/2124712128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f930009d6c0 0x7f930009db40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02a8d0 tx=0x7f92ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.634+0000 7f9306600640 1 -- 172.21.15.88:0/2124712128 reap_dead start 2024-04-26T20:18:38.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.635+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 shutdown_connections 2024-04-26T20:18:38.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.635+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 >> 172.21.15.88:0/2124712128 conn(0x7f93000d2f00 msgr2=0x7f93000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:38.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.635+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 shutdown_connections 2024-04-26T20:18:38.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.635+0000 7f9308600640 1 -- 172.21.15.88:0/2124712128 wait complete. 2024-04-26T20:18:38.654 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T20:18:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.854+0000 7fa2c5000640 1 Processor -- start 2024-04-26T20:18:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.854+0000 7fa2c5000640 1 -- start start 2024-04-26T20:18:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.855+0000 7fa2c5000640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7fa2c015cdf0 0x7fa2c00add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.855+0000 7fa2c5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2c00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2c5000640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c00af9a0 con 0x7fa2c00cd650 2024-04-26T20:18:38.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2c5000640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00ae2c0 con 0x7fa2c015cdf0 2024-04-26T20:18:38.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2c5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00b05b0 con 0x7fa2c006ce80 2024-04-26T20:18:38.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2bd400640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7fa2c00cd650 0x7fa2c006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52182/0 (socket says 172.21.15.88:52182) 2024-04-26T20:18:38.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2bd400640 1 -- 172.21.15.88:0/23844522 learned_addr learned my addr 172.21.15.88:0/23844522 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2be800640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2c00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> v2:172.21.15.132:3300/0 conn(0x7fa2c015cdf0 0x7fa2c00add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607517668 0 0) 0x7fa2c00af9a0 con 0x7fa2c00cd650 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.856+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a4003520 con 0x7fa2c00cd650 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 52605381 0 0) 0x7fa2a4003520 con 0x7fa2c00cd650 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 msgr2=0x7fa2c00b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2c00b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 >> v2:172.21.15.132:3300/0 conn(0x7fa2c015cdf0 msgr2=0x7fa2c00add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 --2- 172.21.15.88:0/23844522 >> v2:172.21.15.132:3300/0 conn(0x7fa2c015cdf0 0x7fa2c00add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00afc80 con 0x7fa2c00cd650 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7fa2ac001670 con 0x7fa2c00cd650 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> v2:172.21.15.132:3300/0 conn(0x7fa2c015cdf0 0x7fa2c00add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 >> v1:172.21.15.88:6789/0 conn(0x7fa2c00cd650 legacy=0x7fa2c006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a40039b0 0x7fa2a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a4004d40 0x7fa2a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b05b0 con 0x7fa2a40039b0 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a400b9f0 con 0x7fa2a4004d40 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a400bbf0 con 0x7fa2c006ce80 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a40039b0 0x7fa2a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bd400640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.857+0000 7fa2bde00640 1 -- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 msgr2=0x7fa2a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.858+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006ce80 0x7fa2a4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.858+0000 7fa2bde00640 1 -- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a4004d40 msgr2=0x7fa2a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:38.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.858+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a4004d40 0x7fa2a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.858+0000 7fa2bde00640 1 -- 172.21.15.88:0/23844522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a4003520 con 0x7fa2a40039b0 2024-04-26T20:18:38.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2bde00640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a40039b0 0x7fa2a4003dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00098a0 tx=0x7fa2a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224e2e1f6c33d872 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000f040 con 0x7fa2a40039b0 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b002ad20 con 0x7fa2a40039b0 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2bca00640 1 -- 172.21.15.88:0/23844522 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0004250 con 0x7fa2a40039b0 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2c5000640 1 -- 172.21.15.88:0/23844522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a40039b0 msgr2=0x7fa2a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2c5000640 1 --2- 172.21.15.88:0/23844522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2a40039b0 0x7fa2a4003dd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00098a0 tx=0x7fa2a4005830 comp rx=0 tx=0).stop 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2be800640 1 -- 172.21.15.88:0/23844522 reap_dead start 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2c5000640 1 -- 172.21.15.88:0/23844522 shutdown_connections 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.859+0000 7fa2c5000640 1 -- 172.21.15.88:0/23844522 >> 172.21.15.88:0/23844522 conn(0x7fa2c00b2d50 msgr2=0x7fa2c00b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.860+0000 7fa2c5000640 1 -- 172.21.15.88:0/23844522 shutdown_connections 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.860+0000 7fa2c5000640 1 -- 172.21.15.88:0/23844522 wait complete. 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.862+0000 7fa2c5000640 1 Processor -- start 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.862+0000 7fa2c5000640 1 -- start start 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a40039b0 0x7fa2c0151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006f480 0x7fa2c006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c015d1d0 con 0x7fa2c01538f0 2024-04-26T20:18:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b0240 con 0x7fa2a40039b0 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2c5000640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c015d350 con 0x7fa2c006f480 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55644/0 (socket says 172.21.15.88:55644) 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 -- 172.21.15.88:0/3834721436 learned_addr learned my addr 172.21.15.88:0/3834721436 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2bd400640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006f480 0x7fa2c006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 -- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006f480 msgr2=0x7fa2c006f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006f480 0x7fa2c006f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 -- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a40039b0 msgr2=0x7fa2c0151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a40039b0 0x7fa2c0151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.863+0000 7fa2be800640 1 -- 172.21.15.88:0/3834721436 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c01542c0 con 0x7fa2c01538f0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2be800640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ee50 tx=0x7fa2b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800bbf0 con 0x7fa2c01538f0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b801a070 con 0x7fa2c01538f0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c009dcb0 con 0x7fa2c01538f0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8014400 con 0x7fa2c01538f0 2024-04-26T20:18:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.864+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c009e270 con 0x7fa2c01538f0 2024-04-26T20:18:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.865+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c0069540 con 0x7fa2c01538f0 2024-04-26T20:18:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.866+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7fa2b8005d10 con 0x7fa2c01538f0 2024-04-26T20:18:38.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.867+0000 7fa2b6000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 0x7fa2a006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:38.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.867+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fa2b808bb60 con 0x7fa2c01538f0 2024-04-26T20:18:38.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.867+0000 7fa2bde00640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 0x7fa2a006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:38.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.868+0000 7fa2bde00640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 0x7fa2a006ef00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0033000 tx=0x7fa2b002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:38.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:38.872+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2c01542c0 con 0x7fa2c01538f0 2024-04-26T20:18:39.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.073+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa2c009da60 con 0x7fa2a006ca40 2024-04-26T20:18:39.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.073+0000 7fa2b6000640 1 -- 172.21.15.88:0/3834721436 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa2c009da60 con 0x7fa2a006ca40 2024-04-26T20:18:39.074 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 msgr2=0x7fa2a006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 0x7fa2a006ef00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0033000 tx=0x7fa2b002c000 comp rx=0 tx=0).stop 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 msgr2=0x7fa2c0153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ee50 tx=0x7fa2b800b480 comp rx=0 tx=0).stop 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 shutdown_connections 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2c01538f0 0x7fa2c0153d80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7fa2a006ca40 0x7fa2a006ef00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7fa2c006f480 0x7fa2c006f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 --2- 172.21.15.88:0/3834721436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2a40039b0 0x7fa2c0151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 >> 172.21.15.88:0/3834721436 conn(0x7fa2c00b2d50 msgr2=0x7fa2c00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:39.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 shutdown_connections 2024-04-26T20:18:39.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.076+0000 7fa2c5000640 1 -- 172.21.15.88:0/3834721436 wait complete. 2024-04-26T20:18:39.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:18:39.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.295+0000 7f879fe00640 1 Processor -- start 2024-04-26T20:18:39.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.295+0000 7f879fe00640 1 -- start start 2024-04-26T20:18:39.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879fe00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f87a00d0e70 0x7f87a00d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879fe00640 1 --2- >> v2:172.21.15.88:3300/0 conn(0x7f87a00b3b20 0x7f87a00b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879fe00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87a0146a60 con 0x7f87a0146680 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879fe00640 1 -- --> v2:172.21.15.88:3300/0 -- mon_getmap magic: 0 -- 0x7f87a006e970 con 0x7f87a00b3b20 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879fe00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f87a00b60a0 con 0x7f87a00d0e70 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879ea00640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f87a0146680 0x7f87a00b5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:52188/0 (socket says 172.21.15.88:52188) 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879ea00640 1 -- 172.21.15.88:0/1828772791 learned_addr learned my addr 172.21.15.88:0/1828772791 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.297+0000 7f879f400640 1 --2- 172.21.15.88:0/1828772791 >> v2:172.21.15.88:3300/0 conn(0x7f87a00b3b20 0x7f87a00b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879e000640 1 --2- 172.21.15.88:0/1828772791 >> v2:172.21.15.132:3300/0 conn(0x7f87a00d0e70 0x7f87a00d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879f400640 1 -- 172.21.15.88:0/1828772791 >> v2:172.21.15.132:3300/0 conn(0x7f87a00d0e70 msgr2=0x7f87a00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:39.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879f400640 1 --2- 172.21.15.88:0/1828772791 >> v2:172.21.15.132:3300/0 conn(0x7f87a00d0e70 0x7f87a00d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879d600640 1 -- 172.21.15.88:0/1828772791 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242841489 0 0) 0x7f87a0146a60 con 0x7f87a0146680 2024-04-26T20:18:39.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879f400640 1 -- 172.21.15.88:0/1828772791 >> v1:172.21.15.88:6789/0 conn(0x7f87a0146680 legacy=0x7f87a00b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.298+0000 7f879f400640 1 -- 172.21.15.88:0/1828772791 --> v2:172.21.15.88:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a006dd60 con 0x7f87a00b3b20 2024-04-26T20:18:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.299+0000 7f879f400640 1 --2- 172.21.15.88:0/1828772791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a00b3f70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f87a0160890 tx=0x7f879802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b506060e68fd97c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.299+0000 7f879d600640 1 -- 172.21.15.88:0/1828772791 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798037070 con 0x7f87a00b3b20 2024-04-26T20:18:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.299+0000 7f879d600640 1 -- 172.21.15.88:0/1828772791 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8798032440 con 0x7f87a00b3b20 2024-04-26T20:18:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.299+0000 7f879d600640 1 -- 172.21.15.88:0/1828772791 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798032810 con 0x7f87a00b3b20 2024-04-26T20:18:39.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 -- 172.21.15.88:0/1828772791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 msgr2=0x7f87a00b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 --2- 172.21.15.88:0/1828772791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a00b3f70 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f87a0160890 tx=0x7f879802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 -- 172.21.15.88:0/1828772791 shutdown_connections 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 --2- 172.21.15.88:0/1828772791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a00b3f70 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 --2- 172.21.15.88:0/1828772791 >> v2:172.21.15.132:3300/0 conn(0x7f87a00d0e70 0x7f87a00d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 -- 172.21.15.88:0/1828772791 >> 172.21.15.88:0/1828772791 conn(0x7f87a006c740 msgr2=0x7f87a006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 -- 172.21.15.88:0/1828772791 shutdown_connections 2024-04-26T20:18:39.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.300+0000 7f879fe00640 1 -- 172.21.15.88:0/1828772791 wait complete. 2024-04-26T20:18:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.302+0000 7f879fe00640 1 Processor -- start 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.302+0000 7f879fe00640 1 -- start start 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00d0e70 0x7f87a00ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0146680 0x7f87a013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87a006e320 con 0x7f87a00b3b20 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f87a00d17f0 con 0x7f87a0146680 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879fe00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f87a006e600 con 0x7f87a00d0e70 2024-04-26T20:18:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55652/0 (socket says 172.21.15.88:55652) 2024-04-26T20:18:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.303+0000 7f879ea00640 1 -- 172.21.15.88:0/1403180040 learned_addr learned my addr 172.21.15.88:0/1403180040 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 -- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00d0e70 msgr2=0x7f87a00ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00d0e70 0x7f87a00ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879f400640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0146680 0x7f87a013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 -- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0146680 msgr2=0x7f87a013aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0146680 0x7f87a013aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 -- 172.21.15.88:0/1403180040 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a015eaa0 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879ea00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f879000c430 tx=0x7f879000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87900074e0 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87900047e0 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8790004bd0 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a015ed80 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.304+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a015f340 con 0x7f87a00b3b20 2024-04-26T20:18:39.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.306+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a00693c0 con 0x7f87a00b3b20 2024-04-26T20:18:39.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.306+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f8790011070 con 0x7f87a00b3b20 2024-04-26T20:18:39.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.307+0000 7f8796a00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 0x7f877c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.307+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f879008b750 con 0x7f87a00b3b20 2024-04-26T20:18:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.308+0000 7f879e000640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 0x7f877c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.308+0000 7f879e000640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 0x7f877c06eed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f87880070b0 tx=0x7f8788007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:39.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.312+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8790059cd0 con 0x7f87a00b3b20 2024-04-26T20:18:39.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.508+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87a0141ad0 con 0x7f877c06ca10 2024-04-26T20:18:39.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.509+0000 7f8796a00640 1 -- 172.21.15.88:0/1403180040 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13353 (secure 0 0 0) 0x7f87a0141ad0 con 0x7f877c06ca10 2024-04-26T20:18:39.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:39.511 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:18:39.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.512+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 msgr2=0x7f877c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.512+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 0x7f877c06eed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f87880070b0 tx=0x7f8788007040 comp rx=0 tx=0).stop 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.512+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 msgr2=0x7f87a015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.512+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f879000c430 tx=0x7f879000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 shutdown_connections 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87a0146680 0x7f87a013aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f877c06ca10 0x7f877c06eed0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f87a00d0e70 0x7f87a00ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 --2- 172.21.15.88:0/1403180040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87a00b3b20 0x7f87a015c370 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:39.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 >> 172.21.15.88:0/1403180040 conn(0x7f87a006c740 msgr2=0x7f87a015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:39.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 shutdown_connections 2024-04-26T20:18:39.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:39.513+0000 7f879fe00640 1 -- 172.21.15.88:0/1403180040 wait complete. 2024-04-26T20:18:39.532 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":633,"stamp":"2024-04-26T20:18:39.339999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261316,"kb_used_data":7688,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713404,"statfs":{"total":773094113280,"available":772826525696,"internally_reserved":0,"allocated":7872512,"data_stored":4044088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.702040"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":269,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T20:18:29.348236+0000","last_change":"2024-04-26T19:59:36.430454+0000","last_active":"2024-04-26T20:18:29.348236+0000","last_peered":"2024-04-26T20:18:29.348236+0000","last_clean":"2024-04-26T20:18:29.348236+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:18:29.348236+0000","last_undegraded":"2024-04-26T20:18:29.348236+0000","last_fullsized":"2024-04-26T20:18:29.348236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:59:35.147627+0000","last_clean_scrub_stamp":"2024-04-26T19:59:35.147627+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:28:57.378445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575080,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:18:39.535 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T19:59:35.147627+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=19, tm_min=59, tm_sec=35, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=18, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T20:18:39.536 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T20:18:51.052 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:18:51.060 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:18:59.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:18:59.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.745+0000 7ff1d6200640 1 Processor -- start 2024-04-26T20:18:59.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.745+0000 7ff1d6200640 1 -- start start 2024-04-26T20:18:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.746+0000 7ff1d6200640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.746+0000 7ff1d6200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.746+0000 7ff1d6200640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d00d0770 con 0x7ff1d01523a0 2024-04-26T20:18:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.746+0000 7ff1d6200640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d006dbc0 con 0x7ff1d0152780 2024-04-26T20:18:59.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.746+0000 7ff1d6200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d006de10 con 0x7ff1d00d0300 2024-04-26T20:18:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.747+0000 7ff1c7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.747+0000 7ff1d5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.747+0000 7ff1d5800640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:47672/0 (socket says 172.21.15.88:47672) 2024-04-26T20:18:59.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.747+0000 7ff1c7e00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51180/0 (socket says 172.21.15.88:51180) 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.747+0000 7ff1d5800640 1 -- 172.21.15.88:0/1154302261 learned_addr learned my addr 172.21.15.88:0/1154302261 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.748+0000 7ff1d5800640 1 -- 172.21.15.88:0/1154302261 >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 msgr2=0x7ff1d015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.748+0000 7ff1d5800640 1 --2- 172.21.15.88:0/1154302261 >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.748+0000 7ff1d5800640 1 -- 172.21.15.88:0/1154302261 >> v1:172.21.15.132:6789/0 conn(0x7ff1d01523a0 legacy=0x7ff1d00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.748+0000 7ff1d5800640 1 -- 172.21.15.88:0/1154302261 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d006e0f0 con 0x7ff1d00d0300 2024-04-26T20:18:59.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.748+0000 7ff1c7e00640 1 --2- 172.21.15.88:0/1154302261 >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:18:59.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.749+0000 7ff1d5800640 1 --2- 172.21.15.88:0/1154302261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800af60 tx=0x7ff1c802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427ee0dba5be9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:59.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.749+0000 7ff1cfe00640 1 -- 172.21.15.88:0/1154302261 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8038070 con 0x7ff1d00d0300 2024-04-26T20:18:59.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.749+0000 7ff1cfe00640 1 -- 172.21.15.88:0/1154302261 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c800a2e0 con 0x7ff1d00d0300 2024-04-26T20:18:59.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.749+0000 7ff1cfe00640 1 -- 172.21.15.88:0/1154302261 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8008720 con 0x7ff1d00d0300 2024-04-26T20:18:59.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 -- 172.21.15.88:0/1154302261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 msgr2=0x7ff1d00b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 --2- 172.21.15.88:0/1154302261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800af60 tx=0x7ff1c802b820 comp rx=0 tx=0).stop 2024-04-26T20:18:59.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 -- 172.21.15.88:0/1154302261 shutdown_connections 2024-04-26T20:18:59.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 --2- 172.21.15.88:0/1154302261 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d00d0300 0x7ff1d00b4570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 --2- 172.21.15.88:0/1154302261 >> v2:172.21.15.132:3300/0 conn(0x7ff1d0152780 0x7ff1d015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 -- 172.21.15.88:0/1154302261 >> 172.21.15.88:0/1154302261 conn(0x7ff1d006c740 msgr2=0x7ff1d006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:59.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 -- 172.21.15.88:0/1154302261 shutdown_connections 2024-04-26T20:18:59.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.750+0000 7ff1d6200640 1 -- 172.21.15.88:0/1154302261 wait complete. 2024-04-26T20:18:59.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.752+0000 7ff1d6200640 1 Processor -- start 2024-04-26T20:18:59.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.752+0000 7ff1d6200640 1 -- start start 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 0x7ff1d00ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1d0152780 0x7ff1d014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00b4ab0 con 0x7ff1d00d0300 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d006e6b0 con 0x7ff1d0152780 2024-04-26T20:18:59.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d6200640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d00b4c30 con 0x7ff1d01523a0 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.753+0000 7ff1d4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44742/0 (socket says 172.21.15.88:44742) 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 -- 172.21.15.88:0/2694846797 learned_addr learned my addr 172.21.15.88:0/2694846797 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1c7e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 0x7ff1d00ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 -- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 msgr2=0x7ff1d00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 0x7ff1d00ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 -- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1d0152780 msgr2=0x7ff1d014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1d0152780 0x7ff1d014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 -- 172.21.15.88:0/2694846797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d014d910 con 0x7ff1d00d0300 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1c7e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 0x7ff1d00ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.754+0000 7ff1d4e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c3c0 tx=0x7ff1b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.755+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40052d0 con 0x7ff1d00d0300 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.755+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b4005490 con 0x7ff1d00d0300 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.755+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4007260 con 0x7ff1d00d0300 2024-04-26T20:18:59.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.755+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d0152cb0 con 0x7ff1d00d0300 2024-04-26T20:18:59.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.755+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d0146190 con 0x7ff1d00d0300 2024-04-26T20:18:59.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.756+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d00693c0 con 0x7ff1d00d0300 2024-04-26T20:18:59.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.757+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7ff1b4007400 con 0x7ff1d00d0300 2024-04-26T20:18:59.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.758+0000 7ff1cd600640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 0x7ff1a406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:18:59.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.758+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7ff1b408ae90 con 0x7ff1d00d0300 2024-04-26T20:18:59.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.758+0000 7ff1c7e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 0x7ff1a406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:18:59.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.759+0000 7ff1c7e00640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 0x7ff1a406ef00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0155d10 tx=0x7ff1bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:18:59.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.763+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1b40594a0 con 0x7ff1d00d0300 2024-04-26T20:18:59.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.960+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 --> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1d00b0bb0 con 0x7ff1a406ca40 2024-04-26T20:18:59.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.961+0000 7ff1cd600640 1 -- 172.21.15.88:0/2694846797 <== mgr.4110 v2:172.21.15.88:6832/318043982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7ff1d00b0bb0 con 0x7ff1a406ca40 2024-04-26T20:18:59.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:18:59.963 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-04-26T20:18:59.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.963+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 msgr2=0x7ff1a406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.963+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 0x7ff1a406ef00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0155d10 tx=0x7ff1bc008040 comp rx=0 tx=0).stop 2024-04-26T20:18:59.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.963+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 msgr2=0x7ff1d0132e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.963+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400c3c0 tx=0x7ff1b400c880 comp rx=0 tx=0).stop 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 shutdown_connections 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1d0152780 0x7ff1d014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7ff1a406ca40 0x7ff1a406ef00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7ff1d01523a0 0x7ff1d00ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 --2- 172.21.15.88:0/2694846797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1d00d0300 0x7ff1d0132e20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:18:59.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 >> 172.21.15.88:0/2694846797 conn(0x7ff1d006c740 msgr2=0x7ff1d00b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:18:59.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 shutdown_connections 2024-04-26T20:18:59.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:18:59.964+0000 7ff1d6200640 1 -- 172.21.15.88:0/2694846797 wait complete. 2024-04-26T20:18:59.984 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":643,"stamp":"2024-04-26T20:18:59.345693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261300,"kb_used_data":7688,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754713420,"statfs":{"total":773094113280,"available":772826542080,"internally_reserved":0,"allocated":7872512,"data_stored":4044088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003473"},"pg_stats":[{"pgid":"1.0","version":"68'119","reported_seq":280,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-26T20:18:39.594443+0000","last_change":"2024-04-26T20:18:39.594574+0000","last_active":"2024-04-26T20:18:39.594443+0000","last_peered":"2024-04-26T20:18:39.594443+0000","last_clean":"2024-04-26T20:18:39.594443+0000","last_became_active":"2024-04-26T19:59:36.428640+0000","last_became_peered":"2024-04-26T19:59:36.428640+0000","last_unstale":"2024-04-26T20:18:39.594443+0000","last_undegraded":"2024-04-26T20:18:39.594443+0000","last_fullsized":"2024-04-26T20:18:39.594443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'119","last_scrub_stamp":"2024-04-26T20:18:39.594208+0000","last_deep_scrub":"68'119","last_deep_scrub_stamp":"2024-04-26T20:18:39.594208+0000","last_clean_scrub_stamp":"2024-04-26T20:18:39.594208+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:31.968005+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575084,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335680,"statfs":{"total":96636764160,"available":96599736320,"internally_reserved":0,"allocated":835584,"data_stored":357919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575084,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1429504,"data_stored":948287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:18:59.985 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T20:19:00.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.175+0000 7f08afc00640 1 Processor -- start 2024-04-26T20:19:00.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.175+0000 7f08afc00640 1 -- start start 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08afc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8072720 0x7f08a815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08afc00640 1 -- --> v1:172.21.15.88:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a80b4ca0 con 0x7f08a80b48c0 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08afc00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a815be80 con 0x7f08a806ce30 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08afc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f08a806d2c0 con 0x7f08a8072720 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08ad200640 1 --1- >> v1:172.21.15.88:6789/0 conn(0x7f08a80b48c0 0x7f08a815f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.88:6789/0 says I am v1:172.21.15.88:47526/0 (socket says 172.21.15.88:47526) 2024-04-26T20:19:00.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.177+0000 7f08ad200640 1 -- 172.21.15.88:0/2729431213 learned_addr learned my addr 172.21.15.88:0/2729431213 (peer_addr_for_me v1:172.21.15.88:0/0) 2024-04-26T20:19:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v1:172.21.15.88:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553912758 0 0) 0x7f0898001110 con 0x7f08a80b48c0 2024-04-26T20:19:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08adc00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8072720 0x7f08a815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 --> v1:172.21.15.88:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f088c004630 con 0x7f08a80b48c0 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v1:172.21.15.88:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 910072096 0 0) 0x7f088c004630 con 0x7f08a80b48c0 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8072720 msgr2=0x7f08a815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8072720 0x7f08a815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 >> v1:172.21.15.132:6789/0 conn(0x7f08a806ce30 legacy=0x7f08a8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 --> v1:172.21.15.88:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a815c160 con 0x7f08a80b48c0 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v1:172.21.15.88:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3295607443 0 0) 0x7f0898001840 con 0x7f08a80b48c0 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 >> v1:172.21.15.88:6789/0 conn(0x7f08a80b48c0 legacy=0x7f08a815f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.178+0000 7f08a7e00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c004af0 0x7f088c004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08a7e00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f088c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08a7e00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c0062d0 0x7f088c003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f088c003980 con 0x7f088c005910 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f088c003b00 con 0x7f088c004af0 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f088c003d00 con 0x7f088c0062d0 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f088c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f08ad200640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c004af0 0x7f088c004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 -- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c0062d0 msgr2=0x7f088c003440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:00.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f088c0062d0 0x7f088c003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 -- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c004af0 msgr2=0x7f088c004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:00.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f088c004af0 0x7f088c004f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.179+0000 7f089fe00640 1 -- 172.21.15.88:0/2729431213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c004630 con 0x7f088c005910 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.180+0000 7f089fe00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f088c005d90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0890001750 tx=0x7f088c006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca780b1df49abde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890001370 con 0x7f088c005910 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08900052d0 con 0x7f088c005910 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08a7e00640 1 -- 172.21.15.88:0/2729431213 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089002abe0 con 0x7f088c005910 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08afc00640 1 -- 172.21.15.88:0/2729431213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 msgr2=0x7f088c005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08afc00640 1 --2- 172.21.15.88:0/2729431213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f088c005d90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0890001750 tx=0x7f088c006940 comp rx=0 tx=0).stop 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08adc00640 1 -- 172.21.15.88:0/2729431213 reap_dead start 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08afc00640 1 -- 172.21.15.88:0/2729431213 shutdown_connections 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08afc00640 1 -- 172.21.15.88:0/2729431213 >> 172.21.15.88:0/2729431213 conn(0x7f08a80b2d50 msgr2=0x7f08a80b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.181+0000 7f08afc00640 1 -- 172.21.15.88:0/2729431213 shutdown_connections 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.182+0000 7f08afc00640 1 -- 172.21.15.88:0/2729431213 wait complete. 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.183+0000 7f08afc00640 1 Processor -- start 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.183+0000 7f08afc00640 1 -- start start 2024-04-26T20:19:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8156530 0x7f08a81569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 0x7f08a815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08a815ee80 con 0x7f088c005910 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f08a8159100 con 0x7f08a815c680 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.184+0000 7f08afc00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f08a815fab0 con 0x7f08a8156530 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44768/0 (socket says 172.21.15.88:44768) 2024-04-26T20:19:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 -- 172.21.15.88:0/1233543460 learned_addr learned my addr 172.21.15.88:0/1233543460 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:19:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08adc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 0x7f08a815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f089fe00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8156530 0x7f08a81569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 -- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8156530 msgr2=0x7f08a81569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8156530 0x7f08a81569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 -- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 msgr2=0x7f08a815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 0x7f08a815cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 -- 172.21.15.88:0/1233543460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a815d370 con 0x7f088c005910 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08adc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 0x7f08a815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.185+0000 7f08ad200640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f089800e860 tx=0x7f089800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.186+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898001810 con 0x7f088c005910 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.186+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0898018070 con 0x7f088c005910 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.186+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800cb10 con 0x7f088c005910 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.186+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a815d650 con 0x7f088c005910 2024-04-26T20:19:00.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.186+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a815dc10 con 0x7f088c005910 2024-04-26T20:19:00.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.187+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a8069650 con 0x7f088c005910 2024-04-26T20:19:00.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.188+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f089800ccb0 con 0x7f088c005910 2024-04-26T20:19:00.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.189+0000 7f08a5600640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 0x7f088406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.189+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f089808b890 con 0x7f088c005910 2024-04-26T20:19:00.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.189+0000 7f089fe00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 0x7f088406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.190+0000 7f089fe00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 0x7f088406ef00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0890032000 tx=0x7f089002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.194+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08a80b4ca0 con 0x7f088c005910 2024-04-26T20:19:00.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.393+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f08a80b3370 con 0x7f088c005910 2024-04-26T20:19:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.399+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f0898059f70 con 0x7f088c005910 2024-04-26T20:19:00.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.400+0000 7f08a5600640 1 -- 172.21.15.88:0/1233543460 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0898059870 con 0x7f088c005910 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 msgr2=0x7f088406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 0x7f088406ef00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0890032000 tx=0x7f089002e000 comp rx=0 tx=0).stop 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 msgr2=0x7f08a809e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f089800e860 tx=0x7f089800ed20 comp rx=0 tx=0).stop 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 shutdown_connections 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a815c680 0x7f08a815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f088406ca40 0x7f088406ef00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f08a8156530 0x7f08a81569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 --2- 172.21.15.88:0/1233543460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f088c005910 0x7f08a809e7a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 >> 172.21.15.88:0/1233543460 conn(0x7f08a80b2d50 msgr2=0x7f08a8072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 shutdown_connections 2024-04-26T20:19:00.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.402+0000 7f08afc00640 1 -- 172.21.15.88:0/1233543460 wait complete. 2024-04-26T20:19:00.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:19:00.424 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:19:00.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.608+0000 7f5829a00640 1 Processor -- start 2024-04-26T20:19:00.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.608+0000 7f5829a00640 1 -- start start 2024-04-26T20:19:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5829a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5829a00640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5829a00640 1 -- --> v1:172.21.15.132:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58240af9a0 con 0x7f58240cd650 2024-04-26T20:19:00.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5829a00640 1 -- --> v2:172.21.15.132:3300/0 -- mon_getmap magic: 0 -- 0x7f58240ae2c0 con 0x7f582406ce80 2024-04-26T20:19:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5829a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f58240b05b0 con 0x7f582415cdf0 2024-04-26T20:19:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5823400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.610+0000 7f5822a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.611+0000 7f5823400640 1 --2- >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51214/0 (socket says 172.21.15.88:51214) 2024-04-26T20:19:00.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.611+0000 7f5822a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3301/0 says I am v2:172.21.15.88:47726/0 (socket says 172.21.15.88:47726) 2024-04-26T20:19:00.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.611+0000 7f5823400640 1 -- 172.21.15.88:0/4056883423 learned_addr learned my addr 172.21.15.88:0/4056883423 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:19:00.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.612+0000 7f5822a00640 1 -- 172.21.15.88:0/4056883423 >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 msgr2=0x7f58240b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:19:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.612+0000 7f5822a00640 1 --2- 172.21.15.88:0/4056883423 >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.612+0000 7f5822a00640 1 -- 172.21.15.88:0/4056883423 >> v1:172.21.15.132:6789/0 conn(0x7f58240cd650 legacy=0x7f582406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.612+0000 7f5822a00640 1 -- 172.21.15.88:0/4056883423 --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58240afc80 con 0x7f582415cdf0 2024-04-26T20:19:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.612+0000 7f5823400640 1 --2- 172.21.15.88:0/4056883423 >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:19:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.613+0000 7f5822a00640 1 --2- 172.21.15.88:0/4056883423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58140097a0 tx=0x7f581402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea4c6718d2fcf2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.613+0000 7f5822000640 1 -- 172.21.15.88:0/4056883423 <== mon.2 v2:172.21.15.132:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814037070 con 0x7f582415cdf0 2024-04-26T20:19:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.613+0000 7f5822000640 1 -- 172.21.15.88:0/4056883423 <== mon.2 v2:172.21.15.132:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5814032440 con 0x7f582415cdf0 2024-04-26T20:19:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.613+0000 7f5822000640 1 -- 172.21.15.88:0/4056883423 <== mon.2 v2:172.21.15.132:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814032830 con 0x7f582415cdf0 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 -- 172.21.15.88:0/4056883423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 msgr2=0x7f58240add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 --2- 172.21.15.88:0/4056883423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f58140097a0 tx=0x7f581402a180 comp rx=0 tx=0).stop 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 -- 172.21.15.88:0/4056883423 shutdown_connections 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 --2- 172.21.15.88:0/4056883423 >> v2:172.21.15.132:3300/0 conn(0x7f582406ce80 0x7f58240b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 --2- 172.21.15.88:0/4056883423 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582415cdf0 0x7f58240add80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 -- 172.21.15.88:0/4056883423 >> 172.21.15.88:0/4056883423 conn(0x7f58240b2d50 msgr2=0x7f58240b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 -- 172.21.15.88:0/4056883423 shutdown_connections 2024-04-26T20:19:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.614+0000 7f5829a00640 1 -- 172.21.15.88:0/4056883423 wait complete. 2024-04-26T20:19:00.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.616+0000 7f5829a00640 1 Processor -- start 2024-04-26T20:19:00.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.617+0000 7f5829a00640 1 -- start start 2024-04-26T20:19:00.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 --2- >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582406ce80 0x7f5824152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 0x7f582409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f582415d1d0 con 0x7f582415cdf0 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 -- 0x7f58240b0240 con 0x7f58240cd650 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5829a00640 1 -- --> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] -- mon_getmap magic: 0 -- 0x7f582415d350 con 0x7f582406ce80 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5823400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5823400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44778/0 (socket says 172.21.15.88:44778) 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.618+0000 7f5823400640 1 -- 172.21.15.88:0/4029410344 learned_addr learned my addr 172.21.15.88:0/4029410344 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-26T20:19:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f581a000640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 0x7f582409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5822a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582406ce80 0x7f5824152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 -- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582406ce80 msgr2=0x7f5824152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582406ce80 0x7f5824152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 -- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 msgr2=0x7f582409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 0x7f582409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 -- 172.21.15.88:0/4029410344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582415f2e0 con 0x7f582415cdf0 2024-04-26T20:19:00.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f581a000640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 0x7f582409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:19:00.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5823400640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ef90 tx=0x7f581c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c00cbe0 con 0x7f582415cdf0 2024-04-26T20:19:00.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f581c00cd80 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.619+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582415f5c0 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.620+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c003bf0 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.620+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5824152ff0 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.621+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824069540 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.622+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 91142+0+0 (secure 0 0 0) 0x7f581c01b030 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.623+0000 7f581b400640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 0x7f580006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.623+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f581c08bff0 con 0x7f582415cdf0 2024-04-26T20:19:00.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.623+0000 7f5822a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 0x7f580006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:19:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.624+0000 7f5822a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 0x7f580006ef00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5814034000 tx=0x7f581402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:19:00.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.628+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f581c022020 con 0x7f582415cdf0 2024-04-26T20:19:00.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.868+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f582406c7e0 con 0x7f582415cdf0 2024-04-26T20:19:00.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.869+0000 7f581b400640 1 -- 172.21.15.88:0/4029410344 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+3872 (secure 0 0 0) 0x7f581c010040 con 0x7f582415cdf0 2024-04-26T20:19:00.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-26T20:19:00.872 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5164,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6834/495673057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":495673057},{"type":"v1","addr":"172.21.15.132:6834","nonce":495673057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":5323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/3109098832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6833","nonce":3109098832},{"type":"v1","addr":"172.21.15.132:6835","nonce":3109098832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2250182867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2250182867},{"type":"v1","addr":"172.21.15.88:6835","nonce":2250182867}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33},{"gid":6141,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/4202377992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":4202377992},{"type":"v1","addr":"172.21.15.88:6837","nonce":4202377992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[]} 2024-04-26T20:19:00.872 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 42 2024-04-26T20:19:00.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 msgr2=0x7f580006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 0x7f580006ef00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5814034000 tx=0x7f581402e000 comp rx=0 tx=0).stop 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 msgr2=0x7f582409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f581c00ef90 tx=0x7f581c00c490 comp rx=0 tx=0).stop 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 shutdown_connections 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f582415cdf0 0x7f582409c9c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240cd650 0x7f582409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.88:6832/318043982,v1:172.21.15.88:6833/318043982] conn(0x7f580006ca40 0x7f580006ef00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 --2- 172.21.15.88:0/4029410344 >> [v2:172.21.15.132:3301/0,v1:172.21.15.132:6790/0] conn(0x7f582406ce80 0x7f5824152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:19:00.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 >> 172.21.15.88:0/4029410344 conn(0x7f58240b2d50 msgr2=0x7f58240b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:19:00.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 shutdown_connections 2024-04-26T20:19:00.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-26T20:19:00.872+0000 7f5829a00640 1 -- 172.21.15.88:0/4029410344 wait complete. 2024-04-26T20:19:00.893 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T20:19:00.894 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T20:19:00.894 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:19:00.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:00.912 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:19:00.912 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:19:00.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:00.988 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:19:00.988 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:19:00.988 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.024 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:19:01.024 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:19:01.025 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.084 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:19:01.085 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T20:19:01.085 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T20:19:01.085 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.163 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T20:19:01.163 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T20:19:01.163 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.225 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T20:19:01.225 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T20:19:01.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.304 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T20:19:01.304 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T20:19:01.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.371 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T20:19:01.371 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T20:19:01.372 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.444 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T20:19:01.445 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T20:19:01.445 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.511 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T20:19:01.512 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T20:19:01.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.570 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T20:19:01.570 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T20:19:01.570 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.643 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T20:19:01.643 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T20:19:01.644 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T20:19:01.644 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:01.729 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T20:19:01.729 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T20:19:01.729 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:06.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-26T20:19:07.831 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T20:19:07.831 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T20:19:07.832 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T20:19:07.832 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:07.887 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T20:19:07.887 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T20:19:07.888 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:07.925 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T20:19:07.925 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T20:19:07.925 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:19:08.002 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T20:19:08.002 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T20:19:08.003 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-04-26T20:19:08.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:19:08.114 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T20:19:08.376 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:19:08.376 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T20:19:08.581 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:19:08.581 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T20:19:08.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:19:08.727 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T20:19:08.892 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:19:08.892 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T20:19:09.142 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:19:09.143 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T20:19:09.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:19:09.273 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T20:19:09.437 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:19:09.438 DEBUG:teuthology.orchestra.run.smithi132:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T20:19:09.611 INFO:tasks.ceph:Archiving mon data... 2024-04-26T20:19:09.613 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/data/mon.a.tgz 2024-04-26T20:19:09.613 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-04-26T20:19:09.670 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.3sAmk54O4O 2024-04-26T20:19:09.670 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3sAmk54O4O 2024-04-26T20:19:09.843 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.3sAmk54O4O 2024-04-26T20:19:09.975 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.3sAmk54O4O is 314KB 2024-04-26T20:19:10.085 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.3sAmk54O4O 2024-04-26T20:19:10.143 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/data/mon.b.tgz 2024-04-26T20:19:10.143 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-26T20:19:10.201 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.2M7dMStEu5 2024-04-26T20:19:10.201 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2M7dMStEu5 2024-04-26T20:19:10.369 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.2M7dMStEu5 2024-04-26T20:19:10.505 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.2M7dMStEu5 is 322KB 2024-04-26T20:19:10.611 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.2M7dMStEu5 2024-04-26T20:19:10.667 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/data/mon.c.tgz 2024-04-26T20:19:10.668 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-04-26T20:19:10.718 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.Ms0Hc5tWjM 2024-04-26T20:19:10.719 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Ms0Hc5tWjM 2024-04-26T20:19:10.875 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.Ms0Hc5tWjM 2024-04-26T20:19:11.014 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.Ms0Hc5tWjM is 322KB 2024-04-26T20:19:11.125 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.Ms0Hc5tWjM 2024-04-26T20:19:11.180 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T20:19:11.181 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:19:11.188 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:19:11.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-26T20:19:11.323 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:19:11.380 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:19:11.381 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T20:19:11.383 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi088/crash 2024-04-26T20:19:11.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:19:11.468 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi132/crash 2024-04-26T20:19:11.469 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:19:11.556 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T20:19:11.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:19:11.564 DEBUG:teuthology.orchestra.run.smithi132:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:19:11.683 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:19:11.766 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:19:11.856 INFO:tasks.ceph:Compressing logs... 2024-04-26T20:19:11.857 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:19:11.865 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:19:11.940 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51492.log 2024-04-26T20:19:11.940 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-04-26T20:19:11.941 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155040.log 2024-04-26T20:19:11.942 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51492.log.gz 2024-04-26T20:19:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-04-26T20:19:11.943 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-04-26T20:19:11.944 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38402.log 2024-04-26T20:19:11.944 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155040.log.gz 2024-04-26T20:19:11.945 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:19:11.945 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-04-26T20:19:11.946 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-04-26T20:19:11.946 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45602.log 2024-04-26T20:19:11.947 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33441.log 2024-04-26T20:19:11.947 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33401.log 2024-04-26T20:19:11.948 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38402.log.gz 2024-04-26T20:19:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155640.log 2024-04-26T20:19:11.948 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2024-04-26T20:19:11.948 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-04-26T20:19:11.949 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45602.log.gz 2024-04-26T20:19:11.949 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-04-26T20:19:11.951 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155640.log.gz 2024-04-26T20:19:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38921.log 2024-04-26T20:19:11.952 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-04-26T20:19:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-04-26T20:19:11.952 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-04-26T20:19:11.952 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-04-26T20:19:11.953 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33441.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:19:11.953 INFO:teuthology.orchestra.run.smithi132.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33441.log.gz 2024-04-26T20:19:11.953 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38854.log 2024-04-26T20:19:11.953 INFO:teuthology.orchestra.run.smithi132.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33401.log.gz 2024-04-26T20:19:11.953 INFO:teuthology.orchestra.run.smithi132.stderr: 93.6% -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph.audit.log: 2024-04-26T20:19:11.954 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T20:19:11.954 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.1.38854.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T20:19:11.954 INFO:teuthology.orchestra.run.smithi132.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.1.38854.log.gz 2024-04-26T20:19:11.956 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T20:19:11.956 INFO:teuthology.orchestra.run.smithi132.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:19:11.956 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T20:19:11.960 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38921.log.gz 2024-04-26T20:19:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38196.log 2024-04-26T20:19:11.960 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-04-26T20:19:11.960 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52668.log 2024-04-26T20:19:11.960 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38196.log.gz 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53953.log 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52668.log.gz 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153968.log 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-04-26T20:19:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51434.log 2024-04-26T20:19:11.962 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53953.log.gz 2024-04-26T20:19:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50134.log 2024-04-26T20:19:11.962 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.153968.log.gz 2024-04-26T20:19:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48209.log 2024-04-26T20:19:11.962 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51434.log.gz 2024-04-26T20:19:11.967 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33481.log 2024-04-26T20:19:11.967 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T20:19:11.967 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33481.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33481.log.gz 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40036.log 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.1.40036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33361.log 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-osd.6.log: 90.2% -- replaced with /var/log/ceph/ceph-client.1.40036.log.gz 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33361.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33361.log.gz 2024-04-26T20:19:11.968 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40695.log 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44540.log 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50134.log.gz 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51870.log 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48209.log.gz 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153764.log 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44540.log.gz 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51870.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27269.log 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51870.log.gz 2024-04-26T20:19:11.969 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155472.log 2024-04-26T20:19:11.970 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153764.log.gz 2024-04-26T20:19:11.970 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.tmp-client.admin.27269.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27269.log.gz 2024-04-26T20:19:11.970 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50701.log 2024-04-26T20:19:11.970 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-04-26T20:19:11.970 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.155472.log.gz 2024-04-26T20:19:11.974 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33230.log 2024-04-26T20:19:11.974 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.1.40695.log: /var/log/ceph/ceph-client.admin.33230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33521.log 88.6% -- replaced with /var/log/ceph/ceph-client.1.40695.log.gz 2024-04-26T20:19:11.974 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-04-26T20:19:11.974 INFO:teuthology.orchestra.run.smithi132.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33230.log.gz 2024-04-26T20:19:11.974 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33321.log 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50701.log.gz 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46391.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155082.log 2024-04-26T20:19:11.976 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40518.log 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48067.log 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155082.log.gz 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40518.log.gz 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48067.log.gz 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52404.log 2024-04-26T20:19:11.977 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-04-26T20:19:11.978 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33521.log.gz 2024-04-26T20:19:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33321.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33321.log.gz 2024-04-26T20:19:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33270.log 2024-04-26T20:19:11.979 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T20:19:11.980 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.admin.33270.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33270.log.gz 2024-04-26T20:19:11.981 INFO:teuthology.orchestra.run.smithi132.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T20:19:11.981 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45044.log 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40139.log 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-04-26T20:19:11.983 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45044.log.gz 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40139.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40139.log.gz 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40602.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-04-26T20:19:11.984 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-04-26T20:19:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41337.log 2024-04-26T20:19:11.987 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.1.41337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39310.log 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53869.log 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1%/var/log/ceph/ceph-client.admin.51146.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-04-26T20:19:11.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51786.log 2024-04-26T20:19:11.989 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53869.log.gz 2024-04-26T20:19:11.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155304.log 2024-04-26T20:19:11.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2024-04-26T20:19:11.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44960.log 2024-04-26T20:19:11.990 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51786.log.gz 2024-04-26T20:19:11.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154422.log 2024-04-26T20:19:11.990 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155304.log.gz 2024-04-26T20:19:11.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53368.log 2024-04-26T20:19:11.991 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44960.log.gz 2024-04-26T20:19:11.992 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2024-04-26T20:19:11.992 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154422.log.gz 2024-04-26T20:19:11.992 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153555.log 2024-04-26T20:19:11.992 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53368.log.gz 2024-04-26T20:19:11.993 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40560.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T20:19:11.993 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2024-04-26T20:19:11.994 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48109.log 2024-04-26T20:19:11.994 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.153555.log.gz 2024-04-26T20:19:11.995 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.3.39310.log: 91.6% -- replaced with /var/log/ceph/ceph-client.3.39310.log.gz 2024-04-26T20:19:11.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47493.log 2024-04-26T20:19:11.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40392.log 2024-04-26T20:19:11.996 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48109.log.gz 2024-04-26T20:19:11.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-04-26T20:19:11.997 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47493.log.gz 2024-04-26T20:19:11.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-04-26T20:19:11.998 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40392.log.gz 2024-04-26T20:19:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38525.log 2024-04-26T20:19:11.999 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-04-26T20:19:12.002 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38525.log.gz 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44876.log 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-04-26T20:19:12.003 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154842.log 2024-04-26T20:19:12.004 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz 2024-04-26T20:19:12.004 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154194.log 2024-04-26T20:19:12.005 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44876.log.gz 2024-04-26T20:19:12.006 INFO:teuthology.orchestra.run.smithi132.stderr: 97.0% -- replaced with /var/log/ceph/ceph-client.1.41337.log.gz 2024-04-26T20:19:12.006 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47779.log 2024-04-26T20:19:12.007 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154842.log.gz 2024-04-26T20:19:12.007 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46619.log 2024-04-26T20:19:12.007 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154194.log.gz 2024-04-26T20:19:12.007 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39131.log 2024-04-26T20:19:12.008 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47779.log.gz 2024-04-26T20:19:12.008 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50284.log 2024-04-26T20:19:12.008 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46619.log.gz 2024-04-26T20:19:12.009 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-04-26T20:19:12.009 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39131.log.gz 2024-04-26T20:19:12.009 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-04-26T20:19:12.010 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50284.log.gz 2024-04-26T20:19:12.010 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42107.log 2024-04-26T20:19:12.011 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-04-26T20:19:12.011 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52141.log 2024-04-26T20:19:12.012 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-04-26T20:19:12.012 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-04-26T20:19:12.013 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42107.log.gz 2024-04-26T20:19:12.013 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52141.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:19:12.014 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52141.log.gz 2024-04-26T20:19:12.015 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-04-26T20:19:12.015 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-04-26T20:19:12.016 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53092.log 2024-04-26T20:19:12.017 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50827.log 2024-04-26T20:19:12.017 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-04-26T20:19:12.017 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:19:12.018 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53176.log 2024-04-26T20:19:12.018 INFO:teuthology.orchestra.run.smithi088.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53092.log.gz 2024-04-26T20:19:12.019 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44624.log 2024-04-26T20:19:12.019 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50827.log.gz 2024-04-26T20:19:12.020 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38738.log 2024-04-26T20:19:12.020 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53176.log.gz 2024-04-26T20:19:12.021 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155850.log 2024-04-26T20:19:12.021 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44624.log.gz 2024-04-26T20:19:12.022 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-04-26T20:19:12.022 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38738.log.gz 2024-04-26T20:19:12.023 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2024-04-26T20:19:12.023 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.155850.log.gz 2024-04-26T20:19:12.024 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51188.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz -5 2024-04-26T20:19:12.024 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-04-26T20:19:12.025 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 2024-04-26T20:19:12.026 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2024-04-26T20:19:12.026 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39439.log 2024-04-26T20:19:12.026 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-04-26T20:19:12.027 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 2024-04-26T20:19:12.027 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-04-26T20:19:12.028 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-04-26T20:19:12.028 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39439.log.gz 2024-04-26T20:19:12.029 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-04-26T20:19:12.029 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz 2024-04-26T20:19:12.030 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33551.log 2024-04-26T20:19:12.030 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-04-26T20:19:12.031 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-04-26T20:19:12.031 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-04-26T20:19:12.032 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.33551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51660.log 2024-04-26T20:19:12.032 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33551.log.gz 2024-04-26T20:19:12.033 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46082.log 2024-04-26T20:19:12.033 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-04-26T20:19:12.034 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-04-26T20:19:12.034 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51660.log.gz 2024-04-26T20:19:12.035 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-04-26T20:19:12.035 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46082.log.gz 2024-04-26T20:19:12.036 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39431.log 2024-04-26T20:19:12.036 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-04-26T20:19:12.037 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154674.log 2024-04-26T20:19:12.037 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-04-26T20:19:12.038 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51912.log 2024-04-26T20:19:12.038 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39431.log.gz 2024-04-26T20:19:12.039 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-04-26T20:19:12.039 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.154674.log.gz 2024-04-26T20:19:12.039 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-04-26T20:19:12.040 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51912.log.gz 2024-04-26T20:19:12.041 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50562.log 2024-04-26T20:19:12.041 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-04-26T20:19:12.042 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49608.log 2024-04-26T20:19:12.042 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-04-26T20:19:12.042 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.50562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-04-26T20:19:12.044 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49608.log: 88.8% -- replaced with /var/log/ceph/ceph-client.0.50562.log.gz 2024-04-26T20:19:12.044 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-04-26T20:19:12.044 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49608.log.gz 2024-04-26T20:19:12.045 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46493.log 2024-04-26T20:19:12.045 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-04-26T20:19:12.046 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48647.log 2024-04-26T20:19:12.046 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-04-26T20:19:12.047 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46493.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.46493.log.gz --verbose 2024-04-26T20:19:12.047 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.52057.log 2024-04-26T20:19:12.048 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48647.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T20:19:12.048 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48647.log.gz 2024-04-26T20:19:12.049 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44152.log 2024-04-26T20:19:12.049 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52057.log.gz 2024-04-26T20:19:12.050 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51020.log 2024-04-26T20:19:12.051 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154590.log 2024-04-26T20:19:12.051 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44152.log.gz 2024-04-26T20:19:12.052 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42821.log 2024-04-26T20:19:12.052 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51020.log.gz 2024-04-26T20:19:12.053 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53785.log 2024-04-26T20:19:12.053 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154590.log.gz 2024-04-26T20:19:12.054 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-04-26T20:19:12.054 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42821.log.gz 2024-04-26T20:19:12.055 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154506.log 2024-04-26T20:19:12.055 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53785.log.gz 2024-04-26T20:19:12.056 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2024-04-26T20:19:12.056 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-04-26T20:19:12.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45441.log 2024-04-26T20:19:12.057 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154506.log.gz 2024-04-26T20:19:12.058 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42905.log 2024-04-26T20:19:12.058 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2024-04-26T20:19:12.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46853.log 2024-04-26T20:19:12.059 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45441.log.gz 2024-04-26T20:19:12.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44582.log 2024-04-26T20:19:12.060 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42905.log.gz 2024-04-26T20:19:12.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38568.log 2024-04-26T20:19:12.062 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46853.log.gz 2024-04-26T20:19:12.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41955.log 2024-04-26T20:19:12.063 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44582.log.gz 2024-04-26T20:19:12.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39005.log 2024-04-26T20:19:12.064 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38568.log.gz 2024-04-26T20:19:12.065 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52309.log 2024-04-26T20:19:12.065 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41955.log.gz 2024-04-26T20:19:12.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43042.log 2024-04-26T20:19:12.066 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39005.log.gz 2024-04-26T20:19:12.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48750.log 2024-04-26T20:19:12.068 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52309.log.gz 2024-04-26T20:19:12.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-04-26T20:19:12.069 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43042.log.gz 2024-04-26T20:19:12.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33283.log 2024-04-26T20:19:12.070 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48750.log.gz 2024-04-26T20:19:12.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-26T20:19:12.071 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-04-26T20:19:12.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.33283.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T20:19:12.072 INFO:teuthology.orchestra.run.smithi088.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33283.log.gz 2024-04-26T20:19:12.073 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155598.log 2024-04-26T20:19:12.073 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-26T20:19:12.074 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53646.log 2024-04-26T20:19:12.075 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155598.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155598.log.gz 2024-04-26T20:19:12.075 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52225.log 2024-04-26T20:19:12.076 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40686.log 2024-04-26T20:19:12.077 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.53646.log: /var/log/ceph/ceph-client.admin.52225.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52225.log.gz 2024-04-26T20:19:12.077 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50032.log 2024-04-26T20:19:12.078 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46577.log 2024-04-26T20:19:12.079 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50032.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40686.log.gz 2024-04-26T20:19:12.080 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45771.log 2024-04-26T20:19:12.080 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50032.log.gz 2024-04-26T20:19:12.081 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46577.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46577.log.gz 2024-04-26T20:19:12.081 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2024-04-26T20:19:12.082 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42149.log 2024-04-26T20:19:12.082 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45771.log.gz 2024-04-26T20:19:12.083 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45855.log 2024-04-26T20:19:12.083 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz 2024-04-26T20:19:12.084 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42149.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42149.log.gz 2024-04-26T20:19:12.084 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48521.log 2024-04-26T20:19:12.085 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45855.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T20:19:12.086 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45855.log.gz 2024-04-26T20:19:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48521.log.gz 2024-04-26T20:19:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T20:19:12.087 INFO:teuthology.orchestra.run.smithi088.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T20:19:12.088 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39774.log 2024-04-26T20:19:12.089 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-04-26T20:19:12.090 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51964.log 2024-04-26T20:19:12.090 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39774.log.gz 2024-04-26T20:19:12.092 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42947.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-04-26T20:19:12.092 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-04-26T20:19:12.093 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51964.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51964.log.gz 2024-04-26T20:19:12.094 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154716.log 2024-04-26T20:19:12.095 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39738.log 2024-04-26T20:19:12.095 INFO:teuthology.orchestra.run.smithi088.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-04-26T20:19:12.096 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154716.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T20:19:12.096 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154716.log.gz 2024-04-26T20:19:12.098 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-04-26T20:19:12.098 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39738.log.gz 2024-04-26T20:19:12.099 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-04-26T20:19:12.100 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47983.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-04-26T20:19:12.102 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.43514.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52752.log 2024-04-26T20:19:12.102 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-04-26T20:19:12.103 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-04-26T20:19:12.104 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52752.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52752.log.gz 2024-04-26T20:19:12.105 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48151.log/var/log/ceph/ceph-client.admin.42485.log: 2024-04-26T20:19:12.106 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-04-26T20:19:12.107 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41228.log 2024-04-26T20:19:12.109 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48151.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48151.log.gz 2024-04-26T20:19:12.109 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log/var/log/ceph/ceph-client.0.41228.log: 2024-04-26T20:19:12.110 INFO:teuthology.orchestra.run.smithi088.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.41228.log.gz 2024-04-26T20:19:12.112 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-04-26T20:19:12.112 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-04-26T20:19:12.114 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47737.log 2024-04-26T20:19:12.115 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49239.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-04-26T20:19:12.116 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.47737.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.155388.log 2024-04-26T20:19:12.116 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47737.log.gz 2024-04-26T20:19:12.117 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44294.log 2024-04-26T20:19:12.119 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155388.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155388.log.gz 2024-04-26T20:19:12.120 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-04-26T20:19:12.120 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44294.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44294.log.gz 2024-04-26T20:19:12.122 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48377.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-04-26T20:19:12.123 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-04-26T20:19:12.124 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43310.log 2024-04-26T20:19:12.126 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-04-26T20:19:12.126 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51266.log 2024-04-26T20:19:12.127 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43310.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43310.log.gz 2024-04-26T20:19:12.128 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54009.log 2024-04-26T20:19:12.129 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51266.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51266.log.gz 2024-04-26T20:19:12.130 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2024-04-26T20:19:12.131 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54009.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54009.log.gz 2024-04-26T20:19:12.132 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40223.log 2024-04-26T20:19:12.133 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44378.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2024-04-26T20:19:12.134 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48707.log 2024-04-26T20:19:12.135 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40223.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40223.log.gz 2024-04-26T20:19:12.136 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50785.log 2024-04-26T20:19:12.137 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48707.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48707.log.gz 2024-04-26T20:19:12.138 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-04-26T20:19:12.138 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50785.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50785.log.gz 2024-04-26T20:19:12.139 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40349.log 2024-04-26T20:19:12.140 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38694.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-04-26T20:19:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40349.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40349.log.gz 2024-04-26T20:19:12.142 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49002.log 2024-04-26T20:19:12.144 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48293.log 2024-04-26T20:19:12.145 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49002.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49002.log.gz 2024-04-26T20:19:12.146 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53218.log 2024-04-26T20:19:12.147 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48293.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48293.log.gz 2024-04-26T20:19:12.148 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45687.log 2024-04-26T20:19:12.149 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53218.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53218.log.gz 2024-04-26T20:19:12.150 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45687.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45687.log.gz 2024-04-26T20:19:12.151 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45897.log 2024-04-26T20:19:12.152 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-04-26T20:19:12.154 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45897.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45897.log.gz 2024-04-26T20:19:12.154 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48605.log 2024-04-26T20:19:12.155 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47905.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-04-26T20:19:12.156 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45254.log 2024-04-26T20:19:12.157 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48605.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48605.log.gz 2024-04-26T20:19:12.158 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155993.log 2024-04-26T20:19:12.159 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45254.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45254.log.gz 2024-04-26T20:19:12.160 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52710.log 2024-04-26T20:19:12.162 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155993.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155993.log.gz 2024-04-26T20:19:12.162 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51576.log 2024-04-26T20:19:12.163 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52710.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52710.log.gz 2024-04-26T20:19:12.164 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39173.log 2024-04-26T20:19:12.165 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51576.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51576.log.gz 2024-04-26T20:19:12.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52796.log 2024-04-26T20:19:12.166 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39173.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39173.log.gz 2024-04-26T20:19:12.168 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155346.log 2024-04-26T20:19:12.170 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52796.log.gz 2024-04-26T20:19:12.170 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39215.log 2024-04-26T20:19:12.171 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155346.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155346.log.gz 2024-04-26T20:19:12.171 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2024-04-26T20:19:12.174 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39215.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39215.log.gz 2024-04-26T20:19:12.174 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48251.log 2024-04-26T20:19:12.174 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-04-26T20:19:12.175 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45729.log 2024-04-26T20:19:12.176 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48251.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48251.log.gz 2024-04-26T20:19:12.177 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42611.log 2024-04-26T20:19:12.178 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45729.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45729.log.gz 2024-04-26T20:19:12.179 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45128.log 2024-04-26T20:19:12.181 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42611.log.gz 2024-04-26T20:19:12.181 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-04-26T20:19:12.181 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45128.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45128.log.gz 2024-04-26T20:19:12.183 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2024-04-26T20:19:12.184 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43892.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-04-26T20:19:12.185 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49990.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2024-04-26T20:19:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T20:19:12.187 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44918.log 2024-04-26T20:19:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.44918.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44918.log.gz 2024-04-26T20:19:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-04-26T20:19:12.193 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-04-26T20:19:12.195 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-04-26T20:19:12.195 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-26T20:19:12.198 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48461.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz/var/log/ceph/ceph-client.admin.49469.log: 2024-04-26T20:19:12.199 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46126.log 2024-04-26T20:19:12.199 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-26T20:19:12.200 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154236.log 2024-04-26T20:19:12.202 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52966.log 2024-04-26T20:19:12.204 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154758.log 2024-04-26T20:19:12.204 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46126.log.gz 2024-04-26T20:19:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154236.log: /var/log/ceph/ceph-client.admin.52966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44834.log 2024-04-26T20:19:12.205 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154236.log.gz 2024-04-26T20:19:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154758.log: /var/log/ceph/ceph-client.admin.44834.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44834.log.gz 2024-04-26T20:19:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48792.log 2024-04-26T20:19:12.209 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52966.log.gz 2024-04-26T20:19:12.209 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154758.log.gz 2024-04-26T20:19:12.213 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2024-04-26T20:19:12.213 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48792.log.gz 2024-04-26T20:19:12.213 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-04-26T20:19:12.214 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50176.log 2024-04-26T20:19:12.216 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44666.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gz 2024-04-26T20:19:12.217 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42653.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-04-26T20:19:12.218 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50176.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50176.log.gz 2024-04-26T20:19:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153848.log 2024-04-26T20:19:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48960.log 2024-04-26T20:19:12.223 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153848.log.gz 2024-04-26T20:19:12.224 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-04-26T20:19:12.229 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48960.log: /var/log/ceph/ceph-client.admin.47863.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50218.log 2024-04-26T20:19:12.229 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48960.log.gz 2024-04-26T20:19:12.230 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-04-26T20:19:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-26T20:19:12.236 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50218.log: /var/log/ceph/ceph-client.admin.46787.log: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154338.log 2024-04-26T20:19:12.238 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52183.log 2024-04-26T20:19:12.238 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50218.log.gz 2024-04-26T20:19:12.239 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-26T20:19:12.242 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52183.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52183.log.gz 2024-04-26T20:19:12.243 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153890.log 2024-04-26T20:19:12.246 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153890.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154338.log.gz 2024-04-26T20:19:12.247 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45212.log 2024-04-26T20:19:12.248 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153890.log.gz 2024-04-26T20:19:12.250 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45212.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45212.log.gz 2024-04-26T20:19:12.251 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51308.log 2024-04-26T20:19:12.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-04-26T20:19:12.255 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51308.log.gz 2024-04-26T20:19:12.256 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42695.log 2024-04-26T20:19:12.261 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43850.log: /var/log/ceph/ceph-client.admin.42695.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154380.log 2024-04-26T20:19:12.262 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 88.5% 2024-04-26T20:19:12.262 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42695.log.gz 2024-04-26T20:19:12.264 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155951.log 2024-04-26T20:19:12.268 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154380.log: /var/log/ceph/ceph-client.admin.155951.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-04-26T20:19:12.269 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154380.log.gz 2024-04-26T20:19:12.270 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155951.log.gz 2024-04-26T20:19:12.271 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-04-26T20:19:12.275 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47312.log: /var/log/ceph/ceph-client.admin.45170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-04-26T20:19:12.275 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-04-26T20:19:12.276 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-04-26T20:19:12.277 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155125.log 2024-04-26T20:19:12.282 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38963.log: /var/log/ceph/ceph-client.admin.155125.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153806.log 2024-04-26T20:19:12.282 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-04-26T20:19:12.283 INFO:teuthology.orchestra.run.smithi132.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T20:19:12.284 INFO:teuthology.orchestra.run.smithi088.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155125.log.gz 2024-04-26T20:19:12.284 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153806.log.gz 2024-04-26T20:19:12.285 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51392.log 2024-04-26T20:19:12.288 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38875.log 2024-04-26T20:19:12.289 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51392.log.gz 2024-04-26T20:19:12.290 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39047.log 2024-04-26T20:19:12.294 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39047.log: /var/log/ceph/ceph-client.admin.38875.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-04-26T20:19:12.295 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38875.log.gz 2024-04-26T20:19:12.296 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39047.log.gz 2024-04-26T20:19:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39436.log 2024-04-26T20:19:12.302 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39436.log: /var/log/ceph/ceph-client.admin.39755.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-04-26T20:19:12.302 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155514.log 2024-04-26T20:19:12.303 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39436.log.gz 2024-04-26T20:19:12.305 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40181.log 2024-04-26T20:19:12.308 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.41809.log 2024-04-26T20:19:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155514.log: 88.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155514.log.gz 2024-04-26T20:19:12.311 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155430.log 2024-04-26T20:19:12.311 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40181.log.gz 2024-04-26T20:19:12.312 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-04-26T20:19:12.316 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.2.41809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155556.log 2024-04-26T20:19:12.316 INFO:teuthology.orchestra.run.smithi088.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.2.41809.log.gz 2024-04-26T20:19:12.317 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.155430.log: --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-04-26T20:19:12.318 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155430.log.gz 2024-04-26T20:19:12.319 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155556.log: /var/log/ceph/ceph-client.admin.43970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2024-04-26T20:19:12.320 INFO:teuthology.orchestra.run.smithi088.stderr: 88.2%/var/log/ceph/ceph-client.admin.42191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39423.log 2024-04-26T20:19:12.321 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42233.log: -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-04-26T20:19:12.321 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.155556.log.gz 2024-04-26T20:19:12.321 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39760.log 2024-04-26T20:19:12.322 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5%/var/log/ceph/ceph-client.admin.39423.log: -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2024-04-26T20:19:12.323 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39423.log.gz 2024-04-26T20:19:12.323 INFO:teuthology.orchestra.run.smithi088.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2024-04-26T20:19:12.325 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154800.log 2024-04-26T20:19:12.327 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39760.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39760.log.gz 2024-04-26T20:19:12.329 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49385.log 2024-04-26T20:19:12.330 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154800.log.gz 2024-04-26T20:19:12.331 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-04-26T20:19:12.335 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49385.log.gz 2024-04-26T20:19:12.335 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33651.log 2024-04-26T20:19:12.336 INFO:teuthology.orchestra.run.smithi132.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T20:19:12.337 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39438.log: gzip 89.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-04-26T20:19:12.337 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.33651.log: -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-04-26T20:19:12.338 INFO:teuthology.orchestra.run.smithi088.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33651.log.gz 2024-04-26T20:19:12.340 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53302.log 2024-04-26T20:19:12.343 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53302.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gz 2024-04-26T20:19:12.343 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48876.log 2024-04-26T20:19:12.344 INFO:teuthology.orchestra.run.smithi088.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53302.log.gz 2024-04-26T20:19:12.345 INFO:teuthology.orchestra.run.smithi088.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T20:19:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-04-26T20:19:12.347 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-04-26T20:19:12.349 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9%/var/log/ceph/ceph-client.admin.45996.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz 2024-04-26T20:19:12.349 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48876.log.gz 2024-04-26T20:19:12.350 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-04-26T20:19:12.350 INFO:teuthology.orchestra.run.smithi088.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-04-26T20:19:12.352 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52484.log 2024-04-26T20:19:12.352 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-04-26T20:19:12.354 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52267.log 2024-04-26T20:19:12.355 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52484.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52484.log.gz 2024-04-26T20:19:12.356 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52267.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52267.log.gz 2024-04-26T20:19:12.357 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53050.log 2024-04-26T20:19:12.358 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43168.log 2024-04-26T20:19:12.360 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53050.log.gz 2024-04-26T20:19:12.361 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43168.log.gz 2024-04-26T20:19:12.361 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-04-26T20:19:12.363 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47539.log 2024-04-26T20:19:12.364 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53008.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-04-26T20:19:12.365 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47539.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47539.log.gz 2024-04-26T20:19:12.366 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2024-04-26T20:19:12.367 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39434.log 2024-04-26T20:19:12.369 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51618.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2024-04-26T20:19:12.370 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39434.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39434.log.gz 2024-04-26T20:19:12.370 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40055.log 2024-04-26T20:19:12.372 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154998.log 2024-04-26T20:19:12.373 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2024-04-26T20:19:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50869.log 2024-04-26T20:19:12.377 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154998.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154998.log.gz 2024-04-26T20:19:12.377 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-04-26T20:19:12.377 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50869.log.gz 2024-04-26T20:19:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46219.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2024-04-26T20:19:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-04-26T20:19:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40265.log 2024-04-26T20:19:12.381 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42737.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gz 2024-04-26T20:19:12.382 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40265.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40265.log.gz 2024-04-26T20:19:12.383 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48025.log 2024-04-26T20:19:12.384 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2024-04-26T20:19:12.386 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48025.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48025.log.gz 2024-04-26T20:19:12.387 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40307.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2024-04-26T20:19:12.388 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-04-26T20:19:12.389 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-04-26T20:19:12.390 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42863.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-04-26T20:19:12.392 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.43394.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-04-26T20:19:12.393 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-04-26T20:19:12.393 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42443.log 2024-04-26T20:19:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51744.log 2024-04-26T20:19:12.395 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49650.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-04-26T20:19:12.396 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42443.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42443.log.gz 2024-04-26T20:19:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51744.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51744.log.gz 2024-04-26T20:19:12.398 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2024-04-26T20:19:12.400 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154464.log 2024-04-26T20:19:12.401 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51350.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-04-26T20:19:12.401 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155220.log 2024-04-26T20:19:12.402 INFO:teuthology.orchestra.run.smithi088.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154464.log.gz 2024-04-26T20:19:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155220.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155220.log.gz 2024-04-26T20:19:12.404 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155808.log 2024-04-26T20:19:12.409 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155808.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155808.log.gz 2024-04-26T20:19:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-04-26T20:19:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49557.log 2024-04-26T20:19:12.410 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49906.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-04-26T20:19:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45521.log 2024-04-26T20:19:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49557.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49557.log.gz 2024-04-26T20:19:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41399.log 2024-04-26T20:19:12.414 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45521.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45521.log.gz 2024-04-26T20:19:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44110.log 2024-04-26T20:19:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41399.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41399.log.gz 2024-04-26T20:19:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44110.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44110.log.gz 2024-04-26T20:19:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51104.log 2024-04-26T20:19:12.419 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42359.log 2024-04-26T20:19:12.421 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51104.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51104.log.gz 2024-04-26T20:19:12.421 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52565.log 2024-04-26T20:19:12.422 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42359.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42359.log.gz 2024-04-26T20:19:12.423 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2024-04-26T20:19:12.424 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52565.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52565.log.gz 2024-04-26T20:19:12.425 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41913.log 2024-04-26T20:19:12.426 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38782.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2024-04-26T20:19:12.427 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40788.log 2024-04-26T20:19:12.428 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41913.log.gz 2024-04-26T20:19:12.429 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40788.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40788.log.gz 2024-04-26T20:19:12.430 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46451.log 2024-04-26T20:19:12.431 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-04-26T20:19:12.433 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46451.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46451.log.gz 2024-04-26T20:19:12.433 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48918.log 2024-04-26T20:19:12.434 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43724.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-04-26T20:19:12.435 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40434.log 2024-04-26T20:19:12.436 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48918.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48918.log.gz 2024-04-26T20:19:12.437 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154010.log 2024-04-26T20:19:12.438 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40434.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40434.log.gz 2024-04-26T20:19:12.439 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153722.log 2024-04-26T20:19:12.440 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154010.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154010.log.gz 2024-04-26T20:19:12.441 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.153722.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153722.log.gz 2024-04-26T20:19:12.442 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-04-26T20:19:12.443 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2024-04-26T20:19:12.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52924.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-04-26T20:19:12.446 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42779.log: /var/log/ceph/ceph-client.admin.40097.log 2024-04-26T20:19:12.447 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2024-04-26T20:19:12.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-04-26T20:19:12.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40097.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-04-26T20:19:12.449 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49343.log 2024-04-26T20:19:12.450 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46307.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-04-26T20:19:12.451 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48834.log 2024-04-26T20:19:12.452 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49343.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49343.log.gz 2024-04-26T20:19:12.453 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45002.log 2024-04-26T20:19:12.453 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48834.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-04-26T20:19:12.454 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154296.log 2024-04-26T20:19:12.456 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45002.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45002.log.gz 2024-04-26T20:19:12.456 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52882.log 2024-04-26T20:19:12.457 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154296.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154296.log.gz 2024-04-26T20:19:12.458 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-04-26T20:19:12.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52882.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52882.log.gz 2024-04-26T20:19:12.460 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47581.log 2024-04-26T20:19:12.460 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-04-26T20:19:12.462 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47581.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47581.log.gz 2024-04-26T20:19:12.463 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45813.log 2024-04-26T20:19:12.464 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45399.log 2024-04-26T20:19:12.466 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45813.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-04-26T20:19:12.466 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47396.log 2024-04-26T20:19:12.467 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45399.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45399.log.gz 2024-04-26T20:19:12.468 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53911.log 2024-04-26T20:19:12.469 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47396.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47396.log.gz 2024-04-26T20:19:12.470 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155178.log 2024-04-26T20:19:12.471 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53911.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53911.log.gz 2024-04-26T20:19:12.472 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39089.log 2024-04-26T20:19:12.473 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155178.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155178.log.gz 2024-04-26T20:19:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39089.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39089.log.gz 2024-04-26T20:19:12.474 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43084.log 2024-04-26T20:19:12.476 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154052.log 2024-04-26T20:19:12.477 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43084.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43084.log.gz 2024-04-26T20:19:12.478 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155724.log 2024-04-26T20:19:12.479 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154052.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154052.log.gz 2024-04-26T20:19:12.480 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2024-04-26T20:19:12.481 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155724.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155724.log.gz 2024-04-26T20:19:12.482 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38652.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2024-04-26T20:19:12.483 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T20:19:12.484 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52099.log 2024-04-26T20:19:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.52099.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52099.log.gz 2024-04-26T20:19:12.488 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155911.log 2024-04-26T20:19:12.490 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49738.log 2024-04-26T20:19:12.492 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155911.log.gz 2024-04-26T20:19:12.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-04-26T20:19:12.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49738.log: /var/log/ceph/ceph-client.admin.48419.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-04-26T20:19:12.497 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49738.log.gz 2024-04-26T20:19:12.499 INFO:teuthology.orchestra.run.smithi088.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-04-26T20:19:12.500 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50925.log 2024-04-26T20:19:12.505 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48335.log: /var/log/ceph/ceph-client.admin.50925.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39752.log 2024-04-26T20:19:12.505 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-04-26T20:19:12.506 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50925.log.gz 2024-04-26T20:19:12.507 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49301.log 2024-04-26T20:19:12.512 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39752.log: /var/log/ceph/ceph-client.admin.49301.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154548.log 2024-04-26T20:19:12.512 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39752.log.gz 2024-04-26T20:19:12.513 INFO:teuthology.orchestra.run.smithi088.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49301.log.gz 2024-04-26T20:19:12.515 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154094.log 2024-04-26T20:19:12.519 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154548.log: /var/log/ceph/ceph-client.admin.154094.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45296.log 2024-04-26T20:19:12.519 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.154548.log.gz 2024-04-26T20:19:12.520 INFO:teuthology.orchestra.run.smithi088.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154094.log.gz 2024-04-26T20:19:12.522 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-04-26T20:19:12.525 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45296.log: /var/log/ceph/ceph-client.admin.43808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46038.log 2024-04-26T20:19:12.526 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-04-26T20:19:12.528 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45296.log.gz 2024-04-26T20:19:12.529 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-04-26T20:19:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46038.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46038.log.gz 2024-04-26T20:19:12.533 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46265.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-04-26T20:19:12.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-04-26T20:19:12.536 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45348.log 2024-04-26T20:19:12.538 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47354.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-04-26T20:19:12.539 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51534.log 2024-04-26T20:19:12.544 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45348.log: /var/log/ceph/ceph-client.admin.51534.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-04-26T20:19:12.544 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45348.log.gz 2024-04-26T20:19:12.545 INFO:teuthology.orchestra.run.smithi088.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51534.log.gz 2024-04-26T20:19:12.546 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50092.log 2024-04-26T20:19:12.550 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43436.log: /var/log/ceph/ceph-client.admin.50092.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45954.log 2024-04-26T20:19:12.551 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-04-26T20:19:12.552 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50092.log.gz 2024-04-26T20:19:12.553 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52015.log 2024-04-26T20:19:12.557 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45954.log: /var/log/ceph/ceph-client.admin.52015.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-04-26T20:19:12.558 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45954.log.gz 2024-04-26T20:19:12.559 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52015.log.gz 2024-04-26T20:19:12.561 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49864.log: -- /var/log/ceph/ceph-client.admin.44210.log 2024-04-26T20:19:12.562 INFO:teuthology.orchestra.run.smithi088.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T20:19:12.564 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz/var/log/ceph/ceph-client.admin.44210.log: 2024-04-26T20:19:12.564 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44210.log.gz 2024-04-26T20:19:12.564 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.40476.log 2024-04-26T20:19:12.565 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-04-26T20:19:12.567 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.40476.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40476.log.gz 2024-04-26T20:19:12.568 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39742.log 2024-04-26T20:19:12.568 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43640.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-04-26T20:19:12.569 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-04-26T20:19:12.570 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39742.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39742.log.gz 2024-04-26T20:19:12.571 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-04-26T20:19:12.572 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-04-26T20:19:12.573 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-04-26T20:19:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43682.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-04-26T20:19:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45086.log 2024-04-26T20:19:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38483.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-04-26T20:19:12.576 INFO:teuthology.orchestra.run.smithi132.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T20:19:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.45086.log: --verbose -- /var/log/ceph/ceph-client.admin.53134.log 2024-04-26T20:19:12.578 INFO:teuthology.orchestra.run.smithi088.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45086.log.gz 2024-04-26T20:19:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155766.log 2024-04-26T20:19:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53134.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53134.log.gz 2024-04-26T20:19:12.581 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48563.log/var/log/ceph/ceph-client.admin.155766.log: 2024-04-26T20:19:12.581 INFO:teuthology.orchestra.run.smithi088.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155766.log.gz 2024-04-26T20:19:12.582 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43210.log 2024-04-26T20:19:12.583 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48563.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48563.log.gz 2024-04-26T20:19:12.584 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43210.log: -- /var/log/ceph/ceph-client.admin.38610.log 2024-04-26T20:19:12.585 INFO:teuthology.orchestra.run.smithi088.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43210.log.gz 2024-04-26T20:19:12.587 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.38610.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38610.log.gz 2024-04-26T20:19:12.587 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154136.log 2024-04-26T20:19:12.589 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47131.log 2024-04-26T20:19:12.590 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154136.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154136.log.gz 2024-04-26T20:19:12.591 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-04-26T20:19:12.592 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.47131.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.47131.log.gz 2024-04-26T20:19:12.593 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46535.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-04-26T20:19:12.594 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.50743.log 2024-04-26T20:19:12.595 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-04-26T20:19:12.596 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50743.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50743.log.gz 2024-04-26T20:19:12.598 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.39437.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-04-26T20:19:12.598 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44498.log 2024-04-26T20:19:12.600 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155262.log 2024-04-26T20:19:12.601 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44498.log.gz 2024-04-26T20:19:12.602 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45644.log 2024-04-26T20:19:12.602 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155262.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155262.log.gz 2024-04-26T20:19:12.603 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49427.log 2024-04-26T20:19:12.604 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45644.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45644.log.gz 2024-04-26T20:19:12.606 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49427.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49427.log.gz 2024-04-26T20:19:12.606 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43268.log 2024-04-26T20:19:12.608 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154632.log 2024-04-26T20:19:12.609 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43268.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43268.log.gz 2024-04-26T20:19:12.609 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155682.log 2024-04-26T20:19:12.614 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.154632.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154632.log.gz 2024-04-26T20:19:12.614 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.155682.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155682.log.gz 2024-04-26T20:19:12.649 INFO:teuthology.orchestra.run.smithi132.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T20:19:12.676 INFO:teuthology.orchestra.run.smithi088.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T20:19:12.715 INFO:teuthology.orchestra.run.smithi088.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T20:19:12.733 INFO:teuthology.orchestra.run.smithi132.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T20:19:12.745 INFO:teuthology.orchestra.run.smithi132.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T20:19:12.858 INFO:teuthology.orchestra.run.smithi088.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T20:19:12.945 INFO:teuthology.orchestra.run.smithi088.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T20:19:13.071 INFO:teuthology.orchestra.run.smithi088.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T20:19:13.837 INFO:teuthology.orchestra.run.smithi088.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.53646.log.gz 2024-04-26T20:19:13.841 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-26T20:19:13.841 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m1.932s 2024-04-26T20:19:13.841 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m5.643s 2024-04-26T20:19:13.842 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m1.408s 2024-04-26T20:19:17.003 INFO:teuthology.orchestra.run.smithi132.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T20:19:17.008 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-04-26T20:19:17.008 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m5.093s 2024-04-26T20:19:17.008 INFO:teuthology.orchestra.run.smithi132.stderr:user 0m8.007s 2024-04-26T20:19:17.008 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m0.750s 2024-04-26T20:19:17.009 INFO:tasks.ceph:Archiving logs... 2024-04-26T20:19:17.009 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi088/log 2024-04-26T20:19:17.010 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:19:18.076 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi132/log 2024-04-26T20:19:18.077 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:19:19.230 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T20:19:19.239 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T20:19:19.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:19:19.248 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:19:19.389 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T20:19:19.397 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T20:19:19.398 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:19:19.406 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:19:19.453 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:19:19.460 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:19:19.463 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:19:19.463 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-26T20:19:19.463 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +1414us[+1411us] +/- 47ms 2024-04-26T20:19:19.463 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 55 -998us[-1001us] +/- 65ms 2024-04-26T20:19:19.463 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -983us[ -983us] +/- 76ms 2024-04-26T20:19:19.464 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:19.469 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:19:19.470 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-04-26T20:19:19.470 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1407us[+1408us] +/- 47ms 2024-04-26T20:19:19.470 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -1070us[-1069us] +/- 65ms 2024-04-26T20:19:19.470 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -994us[ -993us] +/- 76ms 2024-04-26T20:19:19.470 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:19:19.471 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T20:19:19.480 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T20:19:19.481 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T20:19:19.490 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:19:19.703 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:19:19.898 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:19:19.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:19:19.997 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:19:20.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:19:20.090 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T20:19:20.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T20:19:20.116 INFO:teuthology.task.internal:Duration was 1731.731082 seconds 2024-04-26T20:19:20.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T20:19:20.126 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T20:19:20.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:19:20.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:19:20.269 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:19:20.271 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:19:20.640 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T20:19:20.641 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-04-26T20:19:20.641 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:19:20.701 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-04-26T20:19:20.701 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:19:20.775 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T20:19:20.776 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:19:20.783 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:19:20.864 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T20:19:20.865 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:19:20.870 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:19:21.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T20:19:21.058 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T20:19:21.058 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:19:21.141 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:19:21.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T20:19:21.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:19:21.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:19:21.339 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-04-26T20:19:21.343 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-04-26T20:19:21.382 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:19:21.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:19:21.432 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:19:21.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:19:21.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T20:19:21.496 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T20:19:21.497 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi088 2024-04-26T20:19:21.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:19:21.593 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138/remote/smithi132 2024-04-26T20:19:21.593 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:19:21.689 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T20:19:21.689 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:19:21.697 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:19:21.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T20:19:21.764 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T20:19:21.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T20:19:21.773 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T20:19:21.774 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:19:21.779 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:19:21.829 INFO:teuthology.orchestra.run.smithi088.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:19 /home/ubuntu/cephtest 2024-04-26T20:19:21.836 INFO:teuthology.orchestra.run.smithi132.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:19 /home/ubuntu/cephtest 2024-04-26T20:19:21.839 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T20:19:21.849 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T20:19:21.873 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T20:19:21.900 DEBUG:teuthology.nuke:shortname: smithi088 2024-04-26T20:19:21.900 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:19:21.924 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:49.632351', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCGwX4eQ/wpTKEGvn18+qTf2oYqdCO1H0iRhw2c0Gwmi3Hw2hrx6eYf4A0MKCXS3N6up0unUau9jfXFdbuE8VA0='} 2024-04-26T20:19:21.927 DEBUG:teuthology.nuke:shortname: smithi132 2024-04-26T20:19:21.927 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:19:21.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675138', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:49.633901', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzPW8O5pJzgxy81++2UbljKm0/l8yFx5Z+2tD0wPo7w02Ek+rPl0d2yprJHBgcPk27ea05/YSa6Dt/2oVn9H4E='} 2024-04-26T20:19:21.972 INFO:teuthology.orchestra.console.smithi088:Power off 2024-04-26T20:19:21.973 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:19:21.987 INFO:teuthology.orchestra.console.smithi132:Power off 2024-04-26T20:19:21.988 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T20:19:21.998 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-04-26T20:19:21.999 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:22.010 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-04-26T20:19:22.011 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:22.021 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-26T20:19:22.033 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-04-26T20:19:26.023 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:26.034 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:26.134 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-26T20:19:26.143 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-04-26T20:19:30.138 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:30.147 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:30.249 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-26T20:19:30.257 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-04-26T20:19:30.358 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-04-26T20:19:34.250 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T20:19:34.360 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-26T20:19:34.461 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-04-26T20:19:34.566 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 1731.7310824394226 failure_reason: 'Test failure: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes)' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fddcc58813184e8aaa0b94c42a0d0a2f status: fail success: false 2024-04-26T20:19:34.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:19:34.653 INFO:teuthology.run:FAIL